System and method for hybrid processing of construction site images

ABSTRACT

Systems and methods for hybrid processing of construction site images are provided. For example, image data captured from a construction site using at least one image sensor may be obtained. The image data may be analyzed to attempt to recognize an object depicted in the image data. In response to a failure to successfully recognize the object, at least part of the image data may be presented to a user, and a feedback related to the object may be received from the user. For example, the attempt to recognize an object may be based on a construction plan associated with the construction site, and the failure to successfully recognize the object may be identified based on a mismatch between the suggested object type from the attempt to recognize the object and types of objects selected from the construction plan based on the location of the object in the image data.

CROSS REFERENCES TO RELATED APPLICATIONS

This application claims the benefit of priority of U.S. ProvisionalPatent Application No. 62/631,757, filed on Feb. 17, 2018, and U.S.Provisional Patent Application No. 62/666,152, filed on May 3, 2018, andU.S. Provisional Patent Application No. 62/791,841, filed on Jan. 13,2019.

The entire contents of all of the above-identified applications areherein incorporated by reference.

BACKGROUND Technological Field

The disclosed embodiments generally relate to systems and methods forprocessing images. More particularly, the disclosed embodiments relateto systems and methods for processing images of construction siteimages.

Background Information

Image sensors are now part of numerous devices, from security systems tomobile phones, and the availability of images and videos produced bythose devices is increasing.

The construction industry deals with building of new structures,additions and modifications to existing structures, maintenance ofexisting structures, repair of existing structures, improvements ofexisting structures, and so forth. While construction is widespread, theconstruction process still needs improvements. Manual monitoring,analysis, inspection, and management of the construction process proveto be difficult, expensive, and inefficient. As a result, manyconstruction projects suffer from cost and schedule overruns, and inmany times the quality of the constructed structures is lacking.

SUMMARY

In some embodiments, systems comprising at least one processor areprovided. In some examples, the systems may further comprise at leastone of an image sensor, a display device, a communication device, amemory unit, and so forth.

In some embodiments, systems and methods for determining the quality ofconcrete from construction site images are provided.

In some embodiments, image data captured from a construction site usingat least one image sensor may be obtained. The image data may beanalyzed to identify a region of the image data depicting at least partof an object, wherein the object is of an object type and made, at leastpartly, of concrete. The image data may be further analyzed to determinea quality indication associated with the concrete. The object type ofthe object may be used to select a threshold. The quality indication maybe compared with the selected threshold. An indication to a user may beprovided to a user based on a result of the comparison of the qualityindication with the selected threshold.

In some embodiments, systems and methods for providing information basedon construction site images are provided.

In some embodiments, image data captured from a construction site usingat least one image sensor may be obtained. Further, at least oneelectronic record associated with the construction site may be obtained.The image data may be analyzed to identify at least one discrepancybetween the at least one electronic record and the construction site.Further, information based on the identified at least one discrepancymay be provided to a user.

In some embodiments, systems and methods for updating records based onconstruction site images are provided.

In some embodiments, image data captured from a construction site usingat least one image sensor may be obtained. The image data may beanalyzed to detect at least one object in the construction site.Further, at least one electronic record associated with the constructionsite may be updated based on the detected at least one object. In someexamples, the at least one electronic record may comprise a searchabledatabase, and updating the at least one electronic record may compriseindexing the at least one object in the searchable database. Forexample, the searchable database may be searched for a record related tothe at least one object. In response to a determination that thesearchable database includes a record related to the at least oneobject, the record related to the at least one object may be updated. Inresponse to a determination that the searchable database do not includea record related to the at least one object, a record related to the atleast one object may be added to the searchable database.

In some embodiments, systems and methods for generating financialassessments based on construction site images are provided.

In some embodiments, image data captured from a construction site usingat least one image sensor may be obtained. Further, at least oneelectronic record associated with the construction site may be obtained.The image data and the at least one electronic record may be analyzed togenerate at least one financial assessment related to the constructionsite. For example, the image data may be analyzed to identify at leastone discrepancy between the at least one electronic record and theconstruction site, and the identified at least one discrepancy may beused in the generation of the at least one financial assessment.

In some embodiments, systems and methods for hybrid processing ofconstruction site images are provided.

In some embodiments, image data captured from a construction site usingat least one image sensor may be obtained. The image data may beanalyzed to attempt to recognize at least one object depicted in theimage data. In response to a failure to successfully recognize the atleast one object, at least part of the image data may be presented to auser, and a feedback related to the at least one object may be receivedfrom the user. For example, the attempt to recognize the at least oneobject may be based on a construction plan associated with theconstruction site, and the failure to successfully recognize the atleast one object may be identified based on a mismatch between thesuggested object type from the attempt to recognize the at least oneobject and one or more types of one or more objects selected from theconstruction plan based on the location of the at least one object inthe image data.

In some embodiments, systems and methods for ranking entities usingconstruction site images are provided.

In some embodiments, image data captured from a construction site usingat least one image sensor may be obtained. The image data may beanalyzed to detect at least one element depicted in the image data andassociated with an entity. The image data may be further analyzed todetermine at least one property indicative of quality and associatedwith the at least one element. The at least one property may be used togenerate a ranking of the entity. For example, the at least one elementmay include an element built by the entity, installed by the entity,affected by a task performed by the entity, supplied by the entity,manufactured by the entity, and so forth. In some examples, the at leastone property may be based on a discrepancy between a construction planassociated with the construction site and the construction site, betweena project schedule associated with the construction site and theconstruction site, between a financial record associated with theconstruction site and the construction site, between a progress recordassociated with the construction site and the construction site, and soforth.

In some embodiments, systems and methods for annotation of constructionsite images are provided.

In some embodiments, image data captured from a construction site usingat least one image sensor may be obtained. Further, at least oneconstruction plan associated with the construction site and includinginformation related to an object may be obtained. The at least oneconstruction plan may be analyzed to identify a first region of theimage data corresponding to the object. The at least one display devicemay be used to present at least part of the image data to a user with anindication of the identified first region of the image datacorresponding to the object. Further, the at least one display devicemay be used to present to the user a query related to the object. Aresponse to the query may be received from the user. The response may beused to update information associated with the object in at least oneelectronic record associated with the construction site.

Consistent with other disclosed embodiments, non-transitorycomputer-readable storage media may store data and/or computerimplementable instructions for carrying out any of the methods describedherein.

The foregoing general description and the following detailed descriptionare exemplary and explanatory only and are not restrictive of theclaims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are block diagrams illustrating some possibleimplementations of a communicating system.

FIGS. 2A and 2B are block diagrams illustrating some possibleimplementations of an apparatus.

FIG. 3 is a block diagram illustrating a possible implementation of aserver.

FIGS. 4A and 4B are block diagrams illustrating some possibleimplementations of a cloud platform.

FIG. 5 is a block diagram illustrating a possible implementation of acomputational node.

FIG. 6 illustrates an exemplary embodiment of a memory storing aplurality of modules.

FIG. 7 illustrates an example of a method for processing images ofconcrete.

FIG. 8 is a schematic illustration of an example image captured by anapparatus consistent with an embodiment of the present disclosure.

FIG. 9 illustrates an example of a method for providing informationbased on construction site images.

FIG. 10A is a schematic illustration of an example construction planconsistent with an embodiment of the present disclosure.

FIG. 10B is a schematic illustration of an example image captured by anapparatus consistent with an embodiment of the present disclosure.

FIG. 11 illustrates an example of a method for updating records based onconstruction site images.

FIG. 12 illustrates an example of a method for generating financialassessments based on construction site images.

FIG. 13 illustrates an example of a method for hybrid processing ofconstruction site images.

FIG. 14 is a schematic illustration of a user interface consistent withan embodiment of the present disclosure.

FIG. 15 illustrates an example of a method for ranking usingconstruction site images.

FIG. 16 illustrates an example of a method for annotation ofconstruction site images.

FIG. 17 is a schematic illustration of an example image captured by anapparatus consistent with an embodiment of the present disclosure.

DESCRIPTION

Unless specifically stated otherwise, as apparent from the followingdiscussions, it is appreciated that throughout the specificationdiscussions utilizing terms such as “processing”, “calculating”,“computing”, “determining”, “generating”, “setting”, “configuring”,“selecting”, “defining”, “applying”, “obtaining”, “monitoring”,“providing”, “identifying”, “segmenting”, “classifying”, “analyzing”,“associating”, “extracting”, “storing”, “receiving”, “transmitting”, orthe like, include action and/or processes of a computer that manipulateand/or transform data into other data, said data represented as physicalquantities, for example such as electronic quantities, and/or said datarepresenting the physical objects. The terms “computer”, “processor”,“controller”, “processing unit”, “computing unit”, and “processingmodule” should be expansively construed to cover any kind of electronicdevice, component or unit with data processing capabilities, including,by way of non-limiting example, a personal computer, a wearablecomputer, a tablet, a smartphone, a server, a computing system, a cloudcomputing platform, a communication device, a processor, such as, adigital signal processor (DSP), an image signal processor (ISR), amicrocontroller, a field programmable gate array (FPGA), an applicationspecific integrated circuit (ASIC), a central processing unit (CPA), agraphics processing unit (GPU), a visual processing unit (VPU), and soon), possibly with embedded memory, a single core processor, a multicore processor, a core within a processor, any other electroniccomputing device, or any combination of the above.

The operations in accordance with the teachings herein may be performedby a computer specially constructed or programmed to perform thedescribed functions.

As used herein, the phrase “for example,” “such as”, “for instance” andvariants thereof describe non-limiting embodiments of the presentlydisclosed subject matter. Reference in the specification to “one case”,“some cases”, “other cases” or variants thereof means that a particularfeature, structure or characteristic described in connection with theembodiment(s) may be included in at least one embodiment of thepresently disclosed subject matter. Thus the appearance of the phrase“one case”, “some cases”, “other cases” or variants thereof does notnecessarily refer to the same embodiment(s). As used herein, the term“and/or” includes any and all combinations of one or more of theassociated listed items.

It is appreciated that certain features of the presently disclosedsubject matter, which are, for clarity, described in the context ofseparate embodiments, may also be provided in combination in a singleembodiment. Conversely, various features of the presently disclosedsubject matter, which are, for brevity, described in the context of asingle embodiment, may also be provided separately or in any suitablesub-combination.

The term “image sensor” is recognized by those skilled in the art andrefers to any device configured to capture images, a sequence of images,videos, and so forth. This includes sensors that convert optical inputinto images, where optical input can be visible light (like in acamera), radio waves, microwaves, terahertz waves, ultraviolet light,infrared light, x-rays, gamma rays, and/or any other light spectrum.This also includes both 2D and 3D sensors. Examples of image sensortechnologies may include: CCD, CMOS, NMOS, and so forth. 3D sensors maybe implemented using different technologies, including: stereo camera,active stereo camera, time of flight camera, structured light camera,radar, range image camera, and so forth.

The term “compressive strength test” is recognized by those skilled inthe art and refers to a test that mechanically measure the maximalamount of compressive load a material, such as a body or a cube ofconcrete, can bear before fracturing.

The term “water permeability test” is recognized by those skilled in theart and refers to a test of a body or a cube of concrete that measuresthe depth of penetration of water maintained at predetermined pressuresfor a predetermined time intervals.

The term “rapid chloride ion penetration test” is recognized by thoseskilled in the art and refers to a test that measures the ability ofconcrete to resist chloride ion penetration.

The term “water absorption test” is recognized by those skilled in theart and refers to a test of concrete specimens that, after drying thespecimens, emerges the specimens in water at predetermined temperatureand/or pressure for predetermined time intervals, and measures theweight of water absorbed by the specimens.

The term “initial surface absorption test” is recognized by thoseskilled in the art and refers to a test that measures the flow of waterper concrete surface area when subjected to a constant water head.

In embodiments of the presently disclosed subject matter, one or morestages illustrated in the figures may be executed in a different orderand/or one or more groups of stages may be executed simultaneously andvice versa. The figures illustrate a general schematic of the systemarchitecture in accordance embodiments of the presently disclosedsubject matter. Each module in the figures can be made up of anycombination of software, hardware and/or firmware that performs thefunctions as defined and explained herein. The modules in the figuresmay be centralized in one location or dispersed over more than onelocation.

It should be noted that some examples of the presently disclosed subjectmatter are not limited in application to the details of construction andthe arrangement of the components set forth in the following descriptionor illustrated in the drawings. The invention can be capable of otherembodiments or of being practiced or carried out in various ways. Also,it is to be understood that the phraseology and terminology employedherein is for the purpose of description and should not be regarded aslimiting.

In this document, an element of a drawing that is not described withinthe scope of the drawing and is labeled with a numeral that has beendescribed in a previous drawing may have the same use and description asin the previous drawings.

The drawings in this document may not be to any scale. Different figuresmay use different scales and different scales can be used even withinthe same drawing, for example different scales for different views ofthe same object or different scales for the two adjacent objects.

FIG. 1A is a block diagram illustrating a possible implementation of acommunicating system. In this example, apparatuses 200 a and 200 b maycommunicate with server 300 a, with server 300 b, with cloud platform400, with each other, and so forth. Possible implementations ofapparatuses 200 a and 200 b may include apparatus 200 as described inFIGS. 2A and 2B. Possible implementations of servers 300 a and 300 b mayinclude server 300 as described in FIG. 3. Some possible implementationsof cloud platform 400 are described in FIGS. 4A, 4B and 5. In thisexample apparatuses 200 a and 200 b may communicate directly with mobilephone 111, tablet 112, and personal computer (PC) 113. Apparatuses 200 aand 200 b may communicate with local router 120 directly, and/or throughat least one of mobile phone 111, tablet 112, and personal computer (PC)113. In this example, local router 120 may be connected with acommunication network 130. Examples of communication network 130 mayinclude the Internet, phone networks, cellular networks, satellitecommunication networks, private communication networks, virtual privatenetworks (VPN), and so forth. Apparatuses 200 a and 200 b may connect tocommunication network 130 through local router 120 and/or directly.Apparatuses 200 a and 200 b may communicate with other devices, such asservers 300 a, server 300 b, cloud platform 400, remote storage 140 andnetwork attached storage (NAS) 150, through communication network 130and/or directly.

FIG. 1B is a block diagram illustrating a possible implementation of acommunicating system. In this example, apparatuses 200 a, 200 b and 200c may communicate with cloud platform 400 and/or with each other throughcommunication network 130. Possible implementations of apparatuses 200a, 200 b and 200 c may include apparatus 200 as described in FIGS. 2Aand 2B. Some possible implementations of cloud platform 400 aredescribed in FIGS. 4A, 4B and 5.

FIGS. 1A and 1B illustrate some possible implementations of acommunication system. In some embodiments, other communication systemsthat enable communication between apparatus 200 and server 300 may beused. In some embodiments, other communication systems that enablecommunication between apparatus 200 and cloud platform 400 may be used.In some embodiments, other communication systems that enablecommunication among a plurality of apparatuses 200 may be used.

FIG. 2A is a block diagram illustrating a possible implementation ofapparatus 200. In this example, apparatus 200 may comprise: one or morememory units 210, one or more processing units 220, and one or moreimage sensors 260. In some implementations, apparatus 200 may compriseadditional components, while some components listed above may beexcluded.

FIG. 2B is a block diagram illustrating a possible implementation ofapparatus 200. In this example, apparatus 200 may comprise: one or morememory units 210, one or more processing units 220, one or morecommunication modules 230, one or more power sources 240, one or moreaudio sensors 250, one or more image sensors 260, one or more lightsources 265, one or more motion sensors 270, and one or more positioningsensors 275. In some implementations, apparatus 200 may compriseadditional components, while some components listed above may beexcluded. For example, in some implementations apparatus 200 may alsocomprise at least one of the following: one or more barometers; one ormore user input devices; one or more output devices; and so forth. Inanother example, in some implementations at least one of the followingmay be excluded from apparatus 200: memory units 210, communicationmodules 230, power sources 240, audio sensors 250, image sensors 260,light sources 265, motion sensors 270, and positioning sensors 275.

In some embodiments, one or more power sources 240 may be configured to:power apparatus 200; power server 300; power cloud platform 400; and/orpower computational node 500. Possible implementation examples of powersources 240 may include: one or more electric batteries; one or morecapacitors; one or more connections to external power sources; one ormore power convertors; any combination of the above; and so forth.

In some embodiments, the one or more processing units 220 may beconfigured to execute software programs. For example, processing units220 may be configured to execute software programs stored on the memoryunits 210. In some cases, the executed software programs may storeinformation in memory units 210. In some cases, the executed softwareprograms may retrieve information from the memory units 210. Possibleimplementation examples of the processing units 220 may include: one ormore single core processors, one or more multicore processors; one ormore controllers; one or more application processors; one or more systemon a chip processors; one or more central processing units; one or moregraphical processing units; one or more neural processing units; anycombination of the above; and so forth.

In some embodiments, the one or more communication modules 230 may beconfigured to receive and transmit information. For example, controlsignals may be transmitted and/or received through communication modules230. In another example, information received though communicationmodules 230 may be stored in memory units 210. In an additional example,information retrieved from memory units 210 may be transmitted usingcommunication modules 230. In another example, input data may betransmitted and/or received using communication modules 230. Examples ofsuch input data may include: input data inputted by a user using userinput devices; information captured using one or more sensors; and soforth. Examples of such sensors may include: audio sensors 250; imagesensors 260; motion sensors 270; positioning sensors 275; chemicalsensors; temperature sensors; barometers; and so forth.

In some embodiments, the one or more audio sensors 250 may be configuredto capture audio by converting sounds to digital information. Someexamples of audio sensors 250 may include: microphones, unidirectionalmicrophones, bidirectional microphones, cardioid microphones,omnidirectional microphones, onboard microphones, wired microphones,wireless microphones, any combination of the above, and so forth. Insome examples, the captured audio may be stored in memory units 210. Insome additional examples, the captured audio may be transmitted usingcommunication modules 230, for example to other computerized devices,such as server 300, cloud platform 400, computational node 500, and soforth. In some examples, processing units 220 may control the aboveprocesses. For example, processing units 220 may control at least oneof: capturing of the audio; storing the captured audio; transmitting ofthe captured audio; and so forth. In some cases, the captured audio maybe processed by processing units 220. For example, the captured audiomay be compressed by processing units 220; possibly followed: by storingthe compressed captured audio in memory units 210; by transmitted thecompressed captured audio using communication modules 230; and so forth.In another example, the captured audio may be processed using speechrecognition algorithms. In another example, the captured audio may beprocessed using speaker recognition algorithms.

In some embodiments, the one or more image sensors 260 may be configuredto capture visual information by converting light to: images; sequenceof images; videos; 3D images; sequence of 3D images; 3D videos; and soforth. In some examples, the captured visual information may be storedin memory units 210. In some additional examples, the captured visualinformation may be transmitted using communication modules 230, forexample to other computerized devices, such as server 300, cloudplatform 400, computational node 500, and so forth. In some examples,processing units 220 may control the above processes. For example,processing units 220 may control at least one of: capturing of thevisual information; storing the captured visual information;transmitting of the captured visual information; and so forth. In somecases, the captured visual information may be processed by processingunits 220. For example, the captured visual information may becompressed by processing units 220; possibly followed: by storing thecompressed captured visual information in memory units 210; bytransmitted the compressed captured visual information usingcommunication modules 230; and so forth. In another example, thecaptured visual information may be processed in order to: detectobjects, detect events, detect action, detect face, detect people,recognize person, and so forth.

In some embodiments, the one or more light sources 265 may be configuredto emit light, for example in order to enable better image capturing byimage sensors 260. In some examples, the emission of light may becoordinated with the capturing operation of image sensors 260. In someexamples, the emission of light may be continuous. In some examples, theemission of light may be performed at selected times. The emitted lightmay be visible light, infrared light, x-rays, gamma rays, and/or in anyother light spectrum. In some examples, image sensors 260 may capturelight emitted by light sources 265, for example in order to capture 3Dimages and/or 3D videos using active stereo method.

In some embodiments, the one or more motion sensors 270 may beconfigured to perform at least one of the following: detect motion ofobjects in the environment of apparatus 200; measure the velocity ofobjects in the environment of apparatus 200; measure the acceleration ofobjects in the environment of apparatus 200; detect motion of apparatus200; measure the velocity of apparatus 200; measure the acceleration ofapparatus 200; and so forth. In some implementations, the one or moremotion sensors 270 may comprise one or more accelerometers configured todetect changes in proper acceleration and/or to measure properacceleration of apparatus 200. In some implementations, the one or moremotion sensors 270 may comprise one or more gyroscopes configured todetect changes in the orientation of apparatus 200 and/or to measureinformation related to the orientation of apparatus 200. In someimplementations, motion sensors 270 may be implemented using imagesensors 260, for example by analyzing images captured by image sensors260 to perform at least one of the following tasks: track objects in theenvironment of apparatus 200; detect moving objects in the environmentof apparatus 200; measure the velocity of objects in the environment ofapparatus 200; measure the acceleration of objects in the environment ofapparatus 200; measure the velocity of apparatus 200, for example bycalculating the egomotion of image sensors 260; measure the accelerationof apparatus 200, for example by calculating the egomotion of imagesensors 260; and so forth. In some implementations, motion sensors 270may be implemented using image sensors 260 and light sources 265, forexample by implementing a LIDAR using image sensors 260 and lightsources 265. In some implementations, motion sensors 270 may beimplemented using one or more RADARs. In some examples, informationcaptured using motion sensors 270: may be stored in memory units 210,may be processed by processing units 220, may be transmitted and/orreceived using communication modules 230, and so forth.

In some embodiments, the one or more positioning sensors 275 may beconfigured to obtain positioning information of apparatus 200, to detectchanges in the position of apparatus 200, and/or to measure the positionof apparatus 200. In some examples, positioning sensors 275 may beimplemented using one of the following technologies: Global PositioningSystem (GPS), GLObal NAvigation Satellite System (GLONASS), Galileoglobal navigation system, BeiDou navigation system, other GlobalNavigation Satellite Systems (GNSS), Indian Regional NavigationSatellite System (IRNSS), Local Positioning Systems (LPS), Real-TimeLocation Systems (RTLS), Indoor Positioning System (IPS), Wi-Fi basedpositioning systems, cellular triangulation, and so forth. In someexamples, information captured using positioning sensors 275 may bestored in memory units 210, may be processed by processing units 220,may be transmitted and/or received using communication modules 230, andso forth.

In some embodiments, the one or more chemical sensors may be configuredto perform at least one of the following: measure chemical properties inthe environment of apparatus 200; measure changes in the chemicalproperties in the environment of apparatus 200; detect the present ofchemicals in the environment of apparatus 200; measure the concentrationof chemicals in the environment of apparatus 200. Examples of suchchemical properties may include: pH level, toxicity, temperature, and soforth. Examples of such chemicals may include: electrolytes, particularenzymes, particular hormones, particular proteins, smoke, carbondioxide, carbon monoxide, oxygen, ozone, hydrogen, hydrogen sulfide, andso forth. In some examples, information captured using chemical sensorsmay be stored in memory units 210, may be processed by processing units220, may be transmitted and/or received using communication modules 230,and so forth.

In some embodiments, the one or more temperature sensors may beconfigured to detect changes in the temperature of the environment ofapparatus 200 and/or to measure the temperature of the environment ofapparatus 200. In some examples, information captured using temperaturesensors may be stored in memory units 210, may be processed byprocessing units 220, may be transmitted and/or received usingcommunication modules 230, and so forth.

In some embodiments, the one or more barometers may be configured todetect changes in the atmospheric pressure in the environment ofapparatus 200 and/or to measure the atmospheric pressure in theenvironment of apparatus 200. In some examples, information capturedusing the barometers may be stored in memory units 210, may be processedby processing units 220, may be transmitted and/or received usingcommunication modules 230, and so forth.

In some embodiments, the one or more user input devices may beconfigured to allow one or more users to input information. In someexamples, user input devices may comprise at least one of the following:a keyboard, a mouse, a touch pad, a touch screen, a joystick, amicrophone, an image sensor, and so forth. In some examples, the userinput may be in the form of at least one of: text, sounds, speech, handgestures, body gestures, tactile information, and so forth. In someexamples, the user input may be stored in memory units 210, may beprocessed by processing units 220, may be transmitted and/or receivedusing communication modules 230, and so forth.

In some embodiments, the one or more user output devices may beconfigured to provide output information to one or more users. In someexamples, such output information may comprise of at least one of:notifications, feedbacks, reports, and so forth. In some examples, useroutput devices may comprise at least one of: one or more audio outputdevices; one or more textual output devices; one or more visual outputdevices; one or more tactile output devices; and so forth. In someexamples, the one or more audio output devices may be configured tooutput audio to a user, for example through: a headset, a set ofspeakers, and so forth. In some examples, the one or more visual outputdevices may be configured to output visual information to a user, forexample through: a display screen, an augmented reality display system,a printer, a LED indicator, and so forth. In some examples, the one ormore tactile output devices may be configured to output tactilefeedbacks to a user, for example through vibrations, through motions, byapplying forces, and so forth. In some examples, the output may beprovided: in real time, offline, automatically, upon request, and soforth. In some examples, the output information may be read from memoryunits 210, may be provided by a software executed by processing units220, may be transmitted and/or received using communication modules 230,and so forth.

FIG. 3 is a block diagram illustrating a possible implementation ofserver 300. In this example, server 300 may comprise: one or more memoryunits 210, one or more processing units 220, one or more communicationmodules 230, and one or more power sources 240. In some implementations,server 300 may comprise additional components, while some componentslisted above may be excluded. For example, in some implementationsserver 300 may also comprise at least one of the following: one or moreuser input devices; one or more output devices; and so forth. In anotherexample, in some implementations at least one of the following may beexcluded from server 300: memory units 210, communication modules 230,and power sources 240.

FIG. 4A is a block diagram illustrating a possible implementation ofcloud platform 400. In this example, cloud platform 400 may comprisecomputational node 500 a, computational node 500 b, computational node500 c and computational node 500 d. In some examples, a possibleimplementation of computational nodes 500 a, 500 b, 500 c and 500 d maycomprise server 300 as described in FIG. 3. In some examples, a possibleimplementation of computational nodes 500 a, 500 b, 500 c and 500 d maycomprise computational node 500 as described in FIG. 5.

FIG. 4B is a block diagram illustrating a possible implementation ofcloud platform 400. In this example, cloud platform 400 may comprise:one or more computational nodes 500, one or more shared memory modules410, one or more power sources 240, one or more node registrationmodules 420, one or more load balancing modules 430, one or moreinternal communication modules 440, and one or more externalcommunication modules 450. In some implementations, cloud platform 400may comprise additional components, while some components listed abovemay be excluded. For example, in some implementations cloud platform 400may also comprise at least one of the following: one or more user inputdevices; one or more output devices; and so forth. In another example,in some implementations at least one of the following may be excludedfrom cloud platform 400: shared memory modules 410, power sources 240,node registration modules 420, load balancing modules 430, internalcommunication modules 440, and external communication modules 450.

FIG. 5 is a block diagram illustrating a possible implementation ofcomputational node 500. In this example, computational node 500 maycomprise: one or more memory units 210, one or more processing units220, one or more shared memory access modules 510, one or more powersources 240, one or more internal communication modules 440, and one ormore external communication modules 450. In some implementations,computational node 500 may comprise additional components, while somecomponents listed above may be excluded. For example, in someimplementations computational node 500 may also comprise at least one ofthe following: one or more user input devices; one or more outputdevices; and so forth. In another example, in some implementations atleast one of the following may be excluded from computational node 500:memory units 210, shared memory access modules 510, power sources 240,internal communication modules 440, and external communication modules450.

In some embodiments, internal communication modules 440 and externalcommunication modules 450 may be implemented as a combined communicationmodule, such as communication modules 230. In some embodiments, onepossible implementation of cloud platform 400 may comprise server 300.In some embodiments, one possible implementation of computational node500 may comprise server 300. In some embodiments, one possibleimplementation of shared memory access modules 510 may comprise usinginternal communication modules 440 to send information to shared memorymodules 410 and/or receive information from shared memory modules 410.In some embodiments, node registration modules 420 and load balancingmodules 430 may be implemented as a combined module.

In some embodiments, the one or more shared memory modules 410 may beaccessed by more than one computational node. Therefore, shared memorymodules 410 may allow information sharing among two or morecomputational nodes 500. In some embodiments, the one or more sharedmemory access modules 510 may be configured to enable access ofcomputational nodes 500 and/or the one or more processing units 220 ofcomputational nodes 500 to shared memory modules 410. In some examples,computational nodes 500 and/or the one or more processing units 220 ofcomputational nodes 500, may access shared memory modules 410, forexample using shared memory access modules 510, in order to perform atleast one of: executing software programs stored on shared memorymodules 410, store information in shared memory modules 410, retrieveinformation from the shared memory modules 410.

In some embodiments, the one or more node registration modules 420 maybe configured to track the availability of the computational nodes 500.In some examples, node registration modules 420 may be implemented as: asoftware program, such as a software program executed by one or more ofthe computational nodes 500; a hardware solution; a combined softwareand hardware solution; and so forth. In some implementations, noderegistration modules 420 may communicate with computational nodes 500,for example using internal communication modules 440. In some examples,computational nodes 500 may notify node registration modules 420 oftheir status, for example by sending messages: at computational node 500startup; at computational node 500 shutdown; at constant intervals; atselected times; in response to queries received from node registrationmodules 420; and so forth. In some examples, node registration modules420 may query about computational nodes 500 status, for example bysending messages: at node registration module 420 startup; at constantintervals; at selected times; and so forth.

In some embodiments, the one or more load balancing modules 430 may beconfigured to divide the work load among computational nodes 500. Insome examples, load balancing modules 430 may be implemented as: asoftware program, such as a software program executed by one or more ofthe computational nodes 500; a hardware solution; a combined softwareand hardware solution; and so forth. In some implementations, loadbalancing modules 430 may interact with node registration modules 420 inorder to obtain information regarding the availability of thecomputational nodes 500. In some implementations, load balancing modules430 may communicate with computational nodes 500, for example usinginternal communication modules 440. In some examples, computationalnodes 500 may notify load balancing modules 430 of their status, forexample by sending messages: at computational node 500 startup; atcomputational node 500 shutdown; at constant intervals; at selectedtimes; in response to queries received from load balancing modules 430;and so forth. In some examples, load balancing modules 430 may queryabout computational nodes 500 status, for example by sending messages:at load balancing module 430 startup; at constant intervals; at selectedtimes; and so forth.

In some embodiments, the one or more internal communication modules 440may be configured to receive information from one or more components ofcloud platform 400, and/or to transmit information to one or morecomponents of cloud platform 400. For example, control signals and/orsynchronization signals may be sent and/or received through internalcommunication modules 440. In another example, input information forcomputer programs, output information of computer programs, and/orintermediate information of computer programs, may be sent and/orreceived through internal communication modules 440. In another example,information received though internal communication modules 440 may bestored in memory units 210, in shared memory units 410, and so forth. Inan additional example, information retrieved from memory units 210and/or shared memory units 410 may be transmitted using internalcommunication modules 440. In another example, input data may betransmitted and/or received using internal communication modules 440.Examples of such input data may include input data inputted by a userusing user input devices.

In some embodiments, the one or more external communication modules 450may be configured to receive and/or to transmit information. Forexample, control signals may be sent and/or received through externalcommunication modules 450. In another example, information receivedthough external communication modules 450 may be stored in memory units210, in shared memory units 410, and so forth. In an additional example,information retrieved from memory units 210 and/or shared memory units410 may be transmitted using external communication modules 450. Inanother example, input data may be transmitted and/or received usingexternal communication modules 450. Examples of such input data mayinclude: input data inputted by a user using user input devices;information captured from the environment of apparatus 200 using one ormore sensors; and so forth. Examples of such sensors may include: audiosensors 250; image sensors 260; motion sensors 270; positioning sensors275; chemical sensors; temperature sensors; barometers; and so forth.

FIG. 6 illustrates an exemplary embodiment of memory 600 storing aplurality of modules. In some examples, memory 600 may be separate fromand/or integrated with memory units 210, separate from and/or integratedwith memory units 410, and so forth. In some examples, memory 600 may beincluded in a single device, for example in apparatus 200, in server300, in cloud platform 400, in computational node 500, and so forth. Insome examples, memory 600 may be distributed across several devices.Memory 600 may store more or fewer modules than those shown in FIG. 6.In this example, memory 600 may comprise: objects database 605,construction plans 610, as-built models 615, project schedules 620,financial records 625, progress records 630, safety records 635, andconstruction errors 640.

In some embodiments, objects database 605 may comprise informationrelated to objects associated with one or more construction sites. Forexample, the objects may include objects planned to be used in aconstruction site, objects ordered for a construction site, objectsarrived at a construction site and awaiting to be used and/or installed,objects used in a construction site, objects installed in a constructionsite, and so forth. In some examples, the information related to anobject in database 605 may include properties of the object, type,brand, configuration, dimensions, weight, price, supplier, manufacturer,identifier of related construction site, location (for example, withinthe construction site), time of planned arrival, time of actual arrival,time of usage, time of installation, actions need to be taken thatinvolves the object, actions performed using and/or on the object,people associated with the actions (such as persons that need to performan action, persons that performed an action, persons that monitor theaction, persons that approve the action, etc.), tools associated withthe actions (such as tools required to perform an action, tools used toperform the action, etc.), quality, quality of installation, otherobjects used in conjunction with the object, and so forth. In someexamples, elements in objects database 605 may be indexed and/orsearchable, for example using a database, using an indexing datastructure, and so forth.

In some embodiments, construction plans 610 may comprise documents,drawings, models, representations, specifications, measurements, bill ofmaterials, architectural plans, architectural drawings, floor plans, 2Darchitectural plans, 3D architectural plans, construction drawings,feasibility plans, demolition plans, permit plans, mechanical plans,electrical plans, space plans, elevations, sections, renderings,computer-aided design data, Building Information Modeling (BIM) models,and so forth, indicating design intention for one or more constructionsites and/or one or more portions of one or more construction sites.Construction plans 610 may be digitally stored in memory 600, asdescribed above.

In some embodiments, as-built models 615 may comprise documents,drawings, models, representations, specifications, measurements, list ofmaterials, architectural drawings, floor plans, 2D drawings, 3Ddrawings, elevations, sections, renderings, computer-aided design data,Building Information Modeling (BIM) models, and so forth, representingone or more buildings or spaces as they were actually constructed.As-built models 615 may be digitally stored in memory 600, as describedabove.

In some embodiments, project schedules 620 may comprise details ofplanned tasks, milestones, activities, deliverables, expected task starttime, expected task duration, expected task completion date, resourceallocation to tasks, linkages of dependencies between tasks, and soforth, related to one or more construction sites. Project schedules 620may be digitally stored in memory 600, as described above.

In some embodiments, financial records 625 may comprise information,records and documents related to financial transactions, invoices,payment receipts, bank records, work orders, supply orders, deliveryreceipts, rental information, salaries information, financial forecasts,financing details, loans, insurance policies, and so forth, associatedwith one or more construction sites. Financial records 625 may bedigitally stored in memory 600, as described above.

In some embodiments, progress records 630 may comprise information,records and documents related to tasks performed in one or moreconstruction sites, such as actual task start time, actual taskduration, actual task completion date, items used, item affected,resources used, results, and so forth. Progress records 630 may bedigitally stored in memory 600, as described above.

In some embodiments, safety records 635 may include information, recordsand documents related to safety issues (such as hazards, accidents, nearaccidents, safety related events, etc.) associated with one or moreconstruction sites. Safety records 635 may be digitally stored in memory600, as described above.

In some embodiments, construction errors 640 may include information,records and documents related to construction errors (such as executionerrors, divergence from construction plans, improper alignment of items,improper placement or items, improper installation of items, concrete oflow quality, missing item, excess item, and so forth) associated withone or more construction sites. Construction errors 640 may be digitallystored in memory 600, as described above.

In some embodiments, a method, such as methods 700, 900, 1100, 1200,1300, 1500 and 1600, may comprise of one or more steps. In someexamples, these methods, as well as all individual steps therein, may beperformed by various aspects of apparatus 200, server 300, cloudplatform 400, computational node 500, and so forth. For example, asystem comprising of at least one processor, such as processing units220, may perform any of these methods as well as all individual stepstherein, for example by processing units 220 executing softwareinstructions stored within memory units 210 and/or within shared memorymodules 410. In some examples, these methods, as well as all individualsteps therein, may be performed by a dedicated hardware. In someexamples, computer readable medium, such as a non-transitory computerreadable medium, may store data and/or computer implementableinstructions for carrying out any of these methods as well as allindividual steps therein. Some examples of possible execution manners ofa method may include continuous execution (for example, returning to thebeginning of the method once the method normal execution ends),periodically execution, executing the method at selected times,execution upon the detection of a trigger (some examples of such triggermay include a trigger from a user, a trigger from another process, atrigger from an external device, etc.), and so forth.

FIG. 7 illustrates an example of a method 700 for determining thequality of concrete from construction site images. In this example,method 700 may comprise: obtaining image data captured from aconstruction site (Step 710); analyzing the image data to identify aregion depicting an object of an object type and made of concrete (Step720); analyzing the image data to determine a quality indicationassociated with concrete (Step 730); selecting a threshold (Step 740);and comparing the quality indication with the selected threshold (Step750). Based, at least in part, on the result of the comparison, process700 may provide an indication to a user (Step 760). In someimplementations, method 700 may comprise one or more additional steps,while some of the steps listed above may be modified or excluded. Forexample, Step 720 and/or Step 740 and/or Step 750 and/or Step 760 may beexcluded from method 700. In some implementations, one or more stepsillustrated in FIG. 7 may be executed in a different order and/or one ormore groups of steps may be executed simultaneously and vice versa. Forexample, Step 720 may be executed after and/or simultaneously with Step710, Step 730 may be executed after and/or simultaneously with Step 710,Step 730 may be executed before, after and/or simultaneously with Step720, Step 740 may be executed at any stage before Step 750, and soforth.

In some embodiments, obtaining image data captured from a constructionsite (Step 710) may comprise obtaining image data captured from aconstruction site using at least one image sensor, such as image sensors260. In some examples, obtaining the images may comprise capturing theimage data from the construction site. Some examples of image data mayinclude: one or more images; one or more portions of one or more images;sequence of images; one or more video clips; one or more portions of oneor more video clips; one or more video streams; one or more portions ofone or more video streams; one or more 3D images; one or more portionsof one or more 3D images; sequence of 3D images; one or more 3D videoclips; one or more portions of one or more 3D video clips; one or more3D video streams; one or more portions of one or more 3D video streams;one or more 360 images; one or more portions of one or more 360 images;sequence of 360 images; one or more 360 video clips; one or moreportions of one or more 360 video clips; one or more 360 video streams;one or more portions of one or more 360 video streams; informationbased, at least in part, on any of the above; any combination of theabove; and so forth.

In some examples, Step 710 may comprise obtaining image data capturedfrom a construction site (and/or capturing the image data from theconstruction site) using at least one wearable image sensor, such aswearable version of apparatus 200 and/or wearable version of imagesensor 260. For example, the wearable image sensors may be configured tobe worn by construction workers and/or other persons in the constructionsite. For example, the wearable image sensor may be physically connectedand/or integral to a garment, physically connected and/or integral to abelt, physically connected and/or integral to a wrist strap, physicallyconnected and/or integral to a necklace, physically connected and/orintegral to a helmet, and so forth.

In some examples, Step 710 may comprise obtaining image data capturedfrom a construction site (and/or capturing the image data from theconstruction site) using at least one stationary image sensor, such asstationary version of apparatus 200 and/or stationary version of imagesensor 260. For example, the stationary image sensors may be configuredto be mounted to ceilings, to walls, to doorways, to floors, and soforth. For example, a stationary image sensor may be configured to bemounted to a ceiling, for example substantially at the center of theceiling (for example, less than two meters from the center of theceiling, less than one meter from the center of the ceiling, less thanhalf a meter from the center of the ceiling, and so forth), adjunct toan electrical box in the ceiling, at a position in the ceilingcorresponding to a planned connection of a light fixture to the ceiling,and so forth. In another example, two or more stationary image sensorsmay be mounted to a ceiling in a way that ensures that the fields ofview of the two cameras include all walls of the room.

In some examples, Step 710 may comprise obtaining image data capturedfrom a construction site (and/or capturing the image data from theconstruction site) using at least one mobile image sensor, such asmobile version of apparatus 200 and/or mobile version of image sensor260. For example, mobile image sensors may be operated by constructionworkers and/or other persons in the construction site to capture imagedata of the construction site. In another example, mobile image sensorsmay be part of a robot configured to move through the construction siteand capture image data of the construction site. In yet another example,mobile image sensors may be part of a drone configured to fly throughthe construction site and capture image data of the construction site.

In some examples, Step 710 may comprise, in addition or alternatively toobtaining image data and/or other input data, obtaining motioninformation captured using one or more motion sensors, for example usingmotion sensors 270. Examples of such motion information may include:indications related to motion of objects; measurements related to thevelocity of objects; measurements related to the acceleration ofobjects; indications related to motion of motion sensor 270;measurements related to the velocity of motion sensor 270; measurementsrelated to the acceleration of motion sensor 270; information based, atleast in part, on any of the above; any combination of the above; and soforth.

In some examples, Step 710 may comprise, in addition or alternatively toobtaining image data and/or other input data, obtaining positioninformation captured using one or more positioning sensors, for exampleusing positioning sensors 275. Examples of such position information mayinclude: indications related to the position of positioning sensors 275;indications related to changes in the position of positioning sensors275; measurements related to the position of positioning sensors 275;indications related to the orientation of positioning sensors 275;indications related to changes in the orientation of positioning sensors275; measurements related to the orientation of positioning sensors 275;measurements related to changes in the orientation of positioningsensors 275; information based, at least in part, on any of the above;any combination of the above; and so forth.

In some embodiments, Step 710 may comprise receiving input data usingone or more communication devices, such as communication modules 230,internal communication modules 440, external communication modules 450,and so forth. Examples of such input data may include: input datacaptured using one or more sensors; image data captured using imagesensors, for example using image sensors 260; motion informationcaptured using motion sensors, for example using motion sensors 270;position information captured using positioning sensors, for exampleusing positioning sensors 275; and so forth.

In some embodiments, Step 710 may comprise reading input data frommemory units, such as memory units 210, shared memory modules 410, andso forth. Examples of such input data may include: input data capturedusing one or more sensors; image data captured using image sensors, forexample using image sensors 260; motion information captured usingmotion sensors, for example using motion sensors 270; positioninformation captured using positioning sensors, for example usingpositioning sensors 275; and so forth.

In some embodiments, analyzing image data, for example by Step 720, Step730, Step 930, Step 1120, Step 1320, Step 1520, Step 1530, etc., maycomprise analyzing the image data to obtain a preprocessed image data,and subsequently analyzing the image data and/or the preprocessed imagedata to obtain the desired outcome. One of ordinary skill in the artwill recognize that the followings are examples, and that the image datamay be preprocessed using other kinds of preprocessing methods. In someexamples, the image data may be preprocessed by transforming the imagedata using a transformation function to obtain a transformed image data,and the preprocessed image data may comprise the transformed image data.For example, the transformed image data may comprise one or moreconvolutions of the image data. For example, the transformation functionmay comprise one or more image filters, such as low-pass filters,high-pass filters, band-pass filters, all-pass filters, and so forth. Insome examples, the transformation function may comprise a nonlinearfunction. In some examples, the image data may be preprocessed bysmoothing the image data, for example using Gaussian convolution, usinga median filter, and so forth. In some examples, the image data may bepreprocessed to obtain a different representation of the image data. Forexample, the preprocessed image data may comprise: a representation ofat least part of the image data in a frequency domain; a DiscreteFourier Transform of at least part of the image data; a Discrete WaveletTransform of at least part of the image data; a time/frequencyrepresentation of at least part of the image data; a representation ofat least part of the image data in a lower dimension; a lossyrepresentation of at least part of the image data; a losslessrepresentation of at least part of the image data; a time ordered seriesof any of the above; any combination of the above; and so forth. In someexamples, the image data may be preprocessed to extract edges, and thepreprocessed image data may comprise information based on and/or relatedto the extracted edges. In some examples, the image data may bepreprocessed to extract image features from the image data. Someexamples of such image features may comprise information based on and/orrelated to: edges; corners; blobs; ridges; Scale Invariant FeatureTransform (SIFT) features; temporal features; and so forth.

In some embodiments, analyzing image data, for example by Step 720, Step730, Step 930, Step 1120, Step 1320, Step 1520, Step 1530, etc., maycomprise analyzing the image data and/or the preprocessed image datausing one or more rules, functions, procedures, artificial neuralnetworks, object detection algorithms, face detection algorithms, visualevent detection algorithms, action detection algorithms, motiondetection algorithms, background subtraction algorithms, inferencemodels, and so forth. Some examples of such inference models mayinclude: an inference model preprogrammed manually; a classificationmodel; a regression model; a result of training algorithms, such asmachine learning algorithms and/or deep learning algorithms, on trainingexamples, where the training examples may include examples of datainstances, and in some cases, a data instance may be labeled with acorresponding desired label and/or result; and so forth.

In some embodiments, analyzing the image data to identify a regiondepicting an object of an object type and made of concrete (Step 720)may comprise analyzing image data (such as image data captured from aconstruction site using at least one image sensor and obtained by Step710) and/or preprocessed image data to identify a region of the imagedata depicting at least part of an object, wherein the object is of anobject type and made, at least partly, of concrete. In one example,multiple regions may be identified, depicting multiple such objects of asingle object type and made, at least partly, of concrete. In anotherexample, multiple regions may be identified, depicting multiple suchobjects of a plurality of object types and made, at least partly, ofconcrete. In some examples, an identified region of the image data maycomprise rectangular region of the image data containing a depiction ofat least part of the object, map of pixels of the image data containinga depiction of at least part of the object, a single pixel of the imagedata within a depiction of at least part of the object, a continuoussegment of the image data including a depiction of at least part of theobject, a non-continuous segment of the image data including a depictionof at least part of the object, and so forth.

In some examples, the image data may be preprocessed to identify colorsand/or textures within the image data, and a rule for detecting concretebased, at least in part, on the identified colors and/or texture may beused. For example, local histograms of colors and/or textures may beassembled, and concrete may be detected when the assembled histogramsmeet predefined criterions. In some examples, the image data may beprocessed with an inference model to detect regions of concrete. Forexample, the inference model may be a result of a machine learningand/or deep learning algorithm trained on training examples. A trainingexample may comprise example images together with markings of regionsdepicting concrete in the images. The machine learning and/or deeplearning algorithms may be trained using the training examples toidentify images depicting concrete, to identify the regions within theimages that depict concrete, and so forth.

In some examples, the image data may be processed using object detectionalgorithms to identify objects made of concrete, for example to identifyobjects made of concrete of a selected object type. Some examples ofsuch object detection algorithms may include: appearance based objectdetection algorithms, gradient based object detection algorithms, grayscale object detection algorithms, color based object detectionalgorithms, histogram based object detection algorithms, feature basedobject detection algorithms, machine learning based object detectionalgorithms, artificial neural networks based object detectionalgorithms, 2D object detection algorithms, 3D object detectionalgorithms, still image based object detection algorithms, video basedobject detection algorithms, and so forth.

In some examples, Step 720 may further comprise analyzing the image datato determine at least one property related to the detected concrete,such as a size of the surface made of concrete, a color of the concretesurface, a position of the concrete surface (for example based, at leastin part, on the position information and/or motion information obtainedby Step 710), a type of the concrete surface, and so forth. For example,a histogram of the pixel colors and/or gray scale values of theidentified regions of concrete may be generated. In another example, thesize in pixels of the identified regions of concrete may be calculated.In yet another example, the image data may be analyzed to identify atype of the concrete surface, such as an object type (for example, awall, a ceiling, a floor, a stair, and so forth). For example, the imagedata and/or the identified region of the image data may be analyzedusing an inference model configured to determine the type of surface(such as an object type). The inference model may be a result of amachine learning and/or deep learning algorithm trained on trainingexamples. A training example may comprise example images and/or imageregions together with a label describing the type of concrete surface(such as an object type). The inference model may be applied to newimages and/or image regions to determine the type of the surface (suchas an object type).

In some examples, Step 720 may comprise analyzing a construction plan610 associated with the construction site to determine the object typeof the object. For example, the construction plan may be analyzed toidentify an object type specified for an object in the constructionplan, for example based on a position of the object in the constructionsite.

In some examples, Step 720 may comprise analyzing an as-build model 615associated with the construction site to determine the object type ofthe object. For example, the as-build model may be analyzed to identifyan object type specified for an object in the as-build model, forexample based on a position of the object in the construction site.

In some examples, Step 720 may comprise analyzing a project schedule 620associated with the construction site to determine the object type ofthe object. For example, the project schedule may be analyzed toidentify objects of what object types should be in the construction site(or in parts of the construction site) at a certain time (for example,the capturing time of the image data) according to the project schedule.

In some examples, Step 720 may comprise analyzing financial records 625associated with the construction site to determine the object type ofthe object. For example, the financial records may be analyzed toidentify objects of what object types should be in the construction site(or in parts of the construction site) at a certain time (for example,the capturing time of the image data) according to the deliveryreceipts, invoices, purchase orders, and so forth.

In some examples, Step 720 may comprise analyzing progress records 630associated with the construction site to determine the object type ofthe object. For example, the progress records may be analyzed toidentify objects of what object types should be in the construction site(or in parts of the construction site) at a certain time (for example,the capturing time of the image data) according to the progress records.

In some examples, the image data may be analyzed to determine the objecttype of the object of Step 720. For example, the image data may beanalyzed using a machine learning model trained using training examplesto determine object type of an object from one or more images depictingthe object (and/or any other input described above). In another example,the image data may be analyzed by an artificial neural networkconfigured to determine object type of an object from one or more imagesdepicting the object (and/or any other input described above).

In some embodiments, Step 730 may comprise analyzing image data (such asimage data captured from a construction site using at least one imagesensor and obtained by Step 710) and/or preprocessed image data todetermine one or more quality indications associated with the concrete(for example, with concrete depicted in image data captured using Step710, with concrete depicted in regions identified using Step 720, withthe concrete that the object of Step 720 is made of, and so forth). Insome examples, the quality indications may comprise a discrete grade, acontinuous grade, a pass/no pass grade, a degree, a measure, acomparison, and so forth. For example, the quality indication maycomprise an indication of a durability of the concrete. In anotherexample, the quality indication may comprise an indication of strengthof the concrete. In yet another example, the quality indication maycomprise an estimate of a result of a compressive strength testconducted after a selected curing time (such as 28 days, 30 days, 56days, 60 days, one month, two months, and so forth). In another example,the quality indication may comprise an estimate of a result of a waterpermeability test. In yet another example, the quality indication maycomprise an estimate of a result of a rapid chloride ion penetrationtest. In another example, the quality indication may comprise anestimate of a result of a water absorption test. In yet another example,the quality indication may comprise an estimate of a result of aninitial surface absorption test. In some example, the image data may beanalyzed to identify a condition of the concrete, for example where thecondition of the concrete may comprise at least one of segregation ofthe concrete, discoloration of the concrete, scaling of the concrete,crazing of the concrete, cracking of the concrete, and curling of theconcrete. Further, the determination of the quality indication may bebased, at least in part, on the identified condition of the concrete.

In some embodiments, Step 730 may analyze the image data using aninference model to determine quality indications associated withconcrete. For example, the inference model may be a result of a machinelearning and/or deep learning algorithm trained on training examples. Atraining example may comprise example images and/or image regionsdepicting concrete together with desired quality indications. Themachine learning and/or deep learning algorithms may be trained usingthe training examples to generate an inference model that automaticallyproduced quality indications from images of concrete. In some examples,the training examples may comprise images of concrete together with ameasure of the durability of the concrete and/or a measure of thestrength of the concrete (for example as determined by a test conductedon the concrete after the image was captured, as determined by a testconducted on a sample of the concrete, as determined by an expert,etc.), and the machine learning and/or deep learning algorithms may betrained using the training examples to generate an inference model thatautomatically produce a measure of the durability of the concrete and/ora measure of the strength of the concrete from images of concrete. Insome examples, the training examples may comprise images of concretetogether with a result of a test conducted on the concrete after theimage was captured or on a sample of the concrete (such as compressivestrength test, water permeability test, rapid chloride ion penetrationtest, water absorption test, initial surface absorption test, etc.), andthe machine learning and/or deep learning algorithms may be trainedusing the training examples to generate an inference model thatautomatically estimate the result of the test from images of concrete.The above tests may be performed after a selected curing time of theconcrete, such as a day, 36 hours, a week, 28 days, a month, 60 days,less than 30 days, less than 60 days, less than 90 days, more than 28days, more than 56 days, more than 84 days, any combinations of theabove, and so forth. In some examples, the training examples maycomprise images of concrete together with a label indicating a conditionof the concrete (such as ordinary condition, segregation of theconcrete, discoloration of the concrete, scaling of the concrete,crazing of the concrete, cracking of the concrete, curling of theconcrete, etc.), the machine learning and/or deep learning algorithmsmay be trained using the training examples to generate an inferencemodel that automatically identify the condition of concrete from imagesof concrete, and the quality indications may comprise the automaticallyidentified condition of the concrete and/or information based (at leastin part) on the automatically identified condition of the concrete.

In some embodiments, Step 730 may analyze the image data using heuristicrules to determine quality indications associate with concrete. In someexamples, histograms based, at least in part, on the image data and/orregions of the image data may be generated. For example, such histogramsmay comprise histograms of pixel colors, of gray scale values, of imagegradients, of image edges, of image corners, of low level imagefeatures, and so forth. Further, heuristic rules may be used to analyzethe histograms and determine quality indications associate withconcrete. For example, a heuristic rule may specify thresholds fordifferent bins of the histogram, and the heuristic rule may determinethe quality indications associate with concrete based, at least in part,on a comparison of the histogram bin values with the correspondingthresholds, for example by counting the number of bin values that exceedthe corresponding threshold. In some examples, the above thresholds maybe selected based, at least in part, on the type of concrete surface(for example as determined by Step 720), for example using one set ofthreshold values for walls, a second set of threshold values forceilings, a third set of threshold values for stairs, and so forth.

In some embodiments, selecting a threshold (Step 740) may comprise usingthe object type of an object (for example, the object of Step 720) toselect a threshold. For example, in response to a first object type, afirst threshold value may be selected, and in response to a secondobject type, a second threshold value different from the first thresholdvalue may be selected. For example, a lookup table (for example in adatabase) may be used to select a threshold according to an object type.In another example, a regression model configured to take as inputproperties of the object type and calculate a threshold value using theproperties of the object type may be used to select a thresholdaccording to an object type.

In some examples, the selection of the threshold by Step 740 may bebased, at least in part, on quality indications associated with otherobjects. For example, the threshold may be selected to be a function ofthe quality indications associated with the other objects, such as mean,median, mode, minimum, maximum, value that cut the quality indicationsassociated with the other objects to two groups of selected sizes, andso forth. In another example, a distribution of the quality indicationsassociated with other objects may be estimated (for example, using aregression model, using density estimation algorithms, and so forth),and the threshold may be selected to be a function of the estimateddistribution, such as mean, median, standard deviation, variance,coefficient of variation, coefficient of dispersion, a parameter of thebeta-binomial distribution, a property of the distribution (such as amoment of the distribution), any function of the above, and so forth.For example, the distribution may be estimated to as a beta-binomialdistribution, a Wallenius' noncentral hypergeometric distribution, andso forth.

In some examples, the selection of the threshold by Step 740 may bebased, at least in part, on a construction plan associated with theconstruction site. For example, the construction plan may be analyzed toidentify minimal quality indication requirements for one or more objectsmade of concrete, and the threshold may be selected accordingly. In oneexample, the minimal quality indication requirement may be specified inthe construction plan, may be a requirement (such as a legalrequirement, an ordinance requirement, a regulative requirement, anindustry standard requirement, etc.) due to a specific object orconfiguration in the construction plan, and so forth.

In some examples, the object may be within a floor, and the selection ofthe threshold by Step 740 may be based, at least in part, on the floor.For example, the selection of the threshold may be based, at least inpart, on the floor number, the floor height, properties of the floor,and so forth. For example, for an object positioned in a specifiedfloor, a first threshold may be selected, while for an identical orsimilar object positioned in a different specified floor, a secondthreshold different from the first threshold may be selected. Further,the object may be within a building with a number of floors, and theselection of the threshold by Step 740 may be based, at least in part,on the number of floors, on the build height, on properties of thebuilding, and so forth. For example, for an object positioned in aspecified building, a first threshold may be selected, while for anidentical or similar object positioned in a different specifiedbuilding, a second threshold different from the first threshold may beselected. For example, a lookup table (for example in a database) may beused to select a threshold according to properties associated with thefloor and/or the building. In another example, a regression modelconfigured to take as input properties of the floor and/or the buildingand calculate a threshold value using the properties of the floor and/orthe building type may be used to select a threshold according to thefloor and/or the building.

In some examples, the selection of the threshold by Step 740 may bebased, at least in part, on a beam span. For example, for an objectassociated with a first beam span, a first threshold may be selected,while for an identical or similar object associated with a second beamspan, a second threshold different from the first threshold may beselected. For example, the beam span may be compared with a selectedlength, and the selection of the threshold may be based, at least inpart, on a result of the comparison. In another example, a regressionmodel configured to take as input beam span and calculate a thresholdvalue using the beam span may be used to select a threshold according tothe beam span.

In some examples, when the object is a wall of a stairway, the thresholdmay be selected by Step 740 to be a first value, and when the object isa wall not in a stairway, the threshold may be selected by Step 740 tobe a value different than the first value. In some examples, when theobject is part of a lift shaft, the threshold may be selected by Step740 to be a first value, and when the object is not part of a liftshaft, the threshold may be selected by Step 740 to be a value differentthan the first value.

In some examples, the selection of the threshold by Step 740 may bebased, at least in part, on multiple factors. For example, a baselinethreshold may be selected according to an object type as describedabove. Further, in some examples the threshold may be increased ordecreased (for example, by adding or subtracting a selected value, bymultiplying by a selected factor, and so forth) according to at leastone of quality indications associated with other objects in theconstruction site, a construction plan associated with the constructionsite, the floor (for example, properties of the floor as describedabove), the building (for example, properties of the building asdescribed above), and so forth.

In some embodiments, Step 750 may comprise comparing the qualityindication with the selected threshold. For example, a differencebetween a value of the quality indication and the selected threshold maybe calculated. In another example, it may be determined whether thequality indication is higher than the selected threshold or not. In someexamples, an action may be performed based on a result of the comparisonof the quality indication with the selected threshold. For example, inresponse to a first result of the comparison, an action may beperformed, and in response to a second result of the comparison, theperformance of the action may be forgone. In another example, inresponse to a first result of the comparison, a first action may beperformed, and in response to a second result of the comparison, asecond action (different from the first action) may be performed. Someexamples of such actions may include providing an indication to a user(as described below in relation to Step 760), updating an electronicrecord (for example as described below in relation to Step 1130), and soforth.

In some embodiments, Step 760 may comprise providing an indication to auser, for example based, at least in part, on the quality indication(from Step 730) and/or the selected threshold (from Step 740) and/or theresult of the comparison of the quality indication with the selectedthreshold (from Step 750). For example, in response to a first result ofthe comparison, an indication may be provided to the user, and inresponse to a second result of the comparison, the providence of theindication may be forgone. In another example, in response to a firstresult of the comparison, a first indication may be provided to theuser, and in response to a second result of the comparison, a secondindication (different from the first indication) may be provided to theuser. In some examples, the provided indication may comprise apresentation of at least part of the image data with an overlaypresenting information based, at least in part, on the qualityindication (for example, using a display screen, an augmented realitydisplay system, a printer, and so forth). In some examples, indicationsmay be provided to the user when a quality indication fails to meet someselected criterions, when a quality indication do meet some selectedcriterions, and so forth. In some examples, the nature and/or content ofthe indication provided to the user may depend on the quality indicationand/or the region of the image corresponding to the quality indicationsand/or the objects corresponding to the quality indications and/orproperties of the objects (such as position, size, color, object type,and so forth) corresponding to the quality indications. In someexamples, the indications provided to the user may be provided as a:visual output, audio output, tactile output, any combination of theabove, and so forth. In some examples, the amount of indicationsprovided to the user, the events triggering the indications provided tothe user, the content of the indications provided to the user, thenature of the indications provided to the user, etc., may beconfigurable. The indications provided to the user may be provided: bythe apparatus detecting the events, through another apparatus (such as amobile device associated with the user, mobile phone 111, tablet 112,and personal computer 113, etc.), and so forth.

In some embodiments, Step 720 may identify a plurality of regionsdepicting concrete in the image data obtained by Step 710. For eachidentified region, Step 730 may determine quality indications for theconcrete depicted in the region. The quality indications of thedifferent regions may be compared, and information may be presented to auser based, at least in part, on the result of the comparison, forexample as described below. For example, Step 710 may obtain an image ofa staircase made of concrete, Step 720 may identify a region for eachstair, Step 730 may assign quality measure for the concrete of eachstair, the stair corresponding to the lowest quality measure may beidentified, and the identified lowest quality measure may be presentedto the user, for example as an overlay next to the region of the stairin the image. In another example, Step 710 may obtain a 360 degreesimage of a room made of concrete, Step 720 may identify a region foreach wall, Step 730 may assign quality measure for the concrete of eachwall, the wall corresponding to the lowest quality measure may beidentified, and the identified lowest quality measure may be presentedto the user, for example as an overlay on the region of the wall in theimage. In yet another example, Step 710 may obtain video depictingconcrete pillars, Step 720 may identify a frame and/or a region for eachpillar, Step 730 may assign quality measure for the concrete of eachpillar, a selected number of pillars corresponding to the highestquality measures may be identified, and the identified highest qualitymeasures and/or corresponding pillars may be presented to the user.

In some embodiments, Step 720 may identify a region depicting concretein the image data obtained by Step 710, and Step 730 may determinequality indications for the concrete depicted in the region. The qualityindications may be compared with selected thresholds, and informationmay be presented to a user based, at least in part, on the result of thecomparison, for example as described below. In some examples, the abovethresholds may be selected based, at least in part, on the type ofconcrete surface (such as an object type, for example as determined byStep 720), for example using one thresholds for wall, a second thresholdfor ceilings, a third threshold for stairs, and so forth. For example, aquality indication may comprise a measure of the durability of theconcrete and/or a measure of the strength of the concrete, the qualityindication may be compared with a threshold corresponding to a minimaldurability requirement and/or a minimal strength requirement, and anindication may be provided to the user when the measure of durabilityand/or the measure of strength does not meet the minimal requirement. Inanother example, a quality indication may comprise an estimated resultof a test (such as compressive strength test, water permeability test,rapid chloride ion penetration test, water absorption test, initialsurface absorption test, etc.), the quality indication may be comparedwith a threshold corresponding to minimal requirement (for exampleaccording to a standard or regulation), and an indication may beprovided to the user when the estimated result of the test does not meetthe minimal requirement.

FIG. 8 is a schematic illustration of example image 800 captured by anapparatus, such as apparatus 200. Image 800 may depict some objects madeof concrete, such as surface 810, stair 820, stair 830, and wall 840.Method 700 may obtain image 800 using Step 710. As described above, Step720 may identify regions of image 800 depicting objects made ofconcrete, such as concrete surface 810, concrete stair 820, concretestair 830, and concrete wall 840. As described above, Step 730 maydetermine quality indications associated with concrete surface 810,concrete stair 820, concrete stair 830, and concrete wall 840.Information may be provided to a user based, at least in part, on theidentified regions and/or determined quality indications. For example,image 800 may be presented to a user with an overlay specifying theidentified regions and/or determined quality indications. Further, thedetermined quality indications may be compared with selected thresholds,and based on the results of the comparisons, some information may beomitted from the presentation, some information may be presented usingfirst presentation settings (such as font type, font color, font size,background color, emphasis, contrast, transparency, etc.) while otherinformation may be presented using other presentation settings, and soforth. In addition or alternatively to the presentation of image 800, atextual report specifying the identified regions and/or determinedquality indications may be provided to the user.

FIG. 9 illustrates an example of a method 900 for providing informationbased on construction site images. In this example, method 900 maycomprise: obtaining image data captured from a construction site (Step710), obtaining electronic records associated with the construction site(Step 920), analyzing the image data to identify discrepancies betweenthe construction site and the electronic records (Step 930), andproviding information based on the identified discrepancies (Step 940).In some implementations, method 900 may comprise one or more additionalsteps, while some of the steps listed above may be modified or excluded.For example, Step 940 may be excluded from method 900. In someimplementations, one or more steps illustrated in FIG. 9 may be executedin a different order and/or one or more groups of steps may be executedsimultaneously and vice versa. For example, Step 920 may be executedbefore and/or after and/or simultaneously with Step 710, Step 930 may beexecuted after and/or simultaneously with Step 710 and/or Step 920, Step940 may be executed after and/or simultaneously with Step 930, and soforth.

In some embodiments, in Step 920 at least one electronic recordassociated with a construction site may be obtained. For example, the atleast one electronic record obtained by Step 920 may compriseinformation related to objects associated with the construction site,such as objects database 605. In some examples, Step 920 may compriseobtaining at least one electronic construction plan associated with theconstruction site, for example from construction plans 610. In someexamples, Step 920 may comprise obtaining at least one electronicas-built model associated with the construction site, for example fromas-built models 615. In some examples, Step 920 may comprise obtainingat least one electronic project schedule associated with theconstruction site, for example from project schedules 620. In someexamples, Step 920 may comprise obtaining at least one electronicfinancial record associated with the construction site, for example fromfinancial records 625. In some examples, Step 920 may comprise obtainingat least one electronic progress record associated with the constructionsite, for example from progress records 630. In some examples, Step 920may comprise obtaining information related to at least one safety issueassociated with the construction site, for example from safety records635. In some examples, Step 920 may comprise obtaining informationrelated to at least one construction error associated with theconstruction site, for example from construction errors 640.

In some examples, Step 920 may comprise receiving the at least oneelectronic record associated with a construction site using one or morecommunication devices, such as communication modules 230, internalcommunication modules 440, external communication modules 450, and soforth. In some examples, Step 920 may comprise reading the at least oneelectronic record associated with a construction site from memory units,such as memory units 210, shared memory modules 410, and so forth. Insome examples, Step 920 may comprise obtaining information related to atleast one object associated with the construction site, for example fromobjects database 605, by analyzing image data depicting the object inthe construction site (for example using Step 1120 as described below),by analyzing electronic records comprising information about the objectas described below, and so forth. In some examples, Step 920 maycomprise creating the at least one electronic record associated with aconstruction site, for example by using any the methods describedherein. For example, electronic records comprising information relatedto objects in the construction site and made of concrete may be obtainedby using method 700. In another example, electronic records comprisinginformation related to discrepancies between the construction site andother electronic records may be obtained by using method 900. In yetanother example, electronic records comprising information related toobjects in the construction site may be obtained by using method 1100and/or method 1300 and/or method 1600. In another example, electronicrecords comprising information related to financial assessmentsassociated with the construction site may be obtained by using method1200. In yet another example, electronic records comprising informationrelated to entities associated with the construction site may beobtained by using method 1500.

In some embodiments, Step 930 may analyze image data captured from aconstruction site (such as image data captured from the constructionsite using at least one image sensor and obtained by Step 710) toidentify at least one discrepancy between at least one electronic recordassociated with the construction site (such as the at least oneelectronic record obtained by Step 920) and the construction site. Insome examples, Step 930 may analyze the at least one electronic recordand/or the image data using a machine learning model trained usingtraining examples to identify discrepancies between the at least oneelectronic record and the construction site. For example, a trainingexample may comprise an electronic record and image data with acorresponding label detailing discrepancies between the electronicrecord and the construction site. In some examples, Step 930 may analyzethe at least one electronic record and the image data using anartificial neural network configured to identify discrepancies betweenthe at least one electronic record and the construction site.

In some examples, when the at least one electronic record comprises aconstruction plan associated with the construction site (such asconstruction plan 610, construction plan obtained by Step 920, etc.),Step 930 may identify at least one discrepancy between the constructionplan and the construction site. For example, Step 930 may analyze theconstruction plan and/or the image data to identify an object in theconstruction plan that does not exist in the construction site, toidentify an object in the construction site that does not exist in theconstruction plan, to identify an object that has a specified locationaccording to the construction plan and is located at a differentlocation in the construction site (for example, to identify an objectfor which the discrepancy between the location according to theconstruction plan and the location in the construction site is above aselected threshold), to identify an object that should have a specifiedproperty according to the construction plan but has a different propertyin the construction site (some examples of such property may includetype of the object, location of the object, shape of the object,dimensions of the object, color of the object, manufacturer of theobject, type of elements in the object, setting of the object, techniqueof installation of the object, orientation of the object, time of objectinstallment, etc.), to identify an object that should be associated witha specified quantity according to the construction plan but isassociated with a different quantity in the construction site (someexamples of such quantities may include size of the object, dimensionsof the object, number of elements in the object, etc.), and so forth.For example, the image data may be analyzed to detect objects and/or todetermine properties of the detected objects (for example, using Step1120 as described below), the detected objects may be searched in theconstruction plan (for example using the determined properties), andStep 930 may identify objects detect in the image data that are notfound in the construction plan as a discrepancies. In another example,the construction plan may be analyzed to identify objects and/orproperties of the identified objects, the identified objects may besearched in the image data (for example, as described above, using theidentified properties, etc.), and Step 930 may identify objectsidentified in the construction plan that are not found in the image dataas discrepancies. In yet another example, objects found both in theimage data (for example, as described above) and in the constructionplan (for example, as described above) may be identified, and Step 930may compare properties of the identified objects in the image data (forexample, determined as described above) with properties of theidentified objects in the construction plan to identify discrepancies.Some examples of such properties may include location of the object,quantity associated with the object (as described above), type of theobject, shape of the object, dimensions of the object, color of theobject, manufacturer of the object, type of elements in the object,setting of the object, technique of installation of the object,orientation of the object, time of object installment, and so forth.

In some examples, when the at least one electronic record comprises aproject schedule associated with the construction site (such as projectschedule 620, project schedule obtained by Step 920, etc.), Step 930 mayidentify at least one discrepancy between the project schedule and theconstruction site. For example, the image data may be associated withtime (for example, the capturing time of the image data, the receivingtime of the image data, the time of processing of the image data, etc.),and Step 930 may identify at least one discrepancy between a desiredstate of the construction site at the associated time according to theproject schedule and the state of the actual construction site at theassociated time as depicted in the image data. For example, the projectschedule and/or the image data may be analyzed to identify an object inthe construction site at a certain time that should not be in theconstruction site at the certain time according to the project schedule,to identify an object that should be in the construction site at acertain time according to the project schedule that is not in theconstruction site at the certain time, to identify an object in theconstruction site that is in a first state at a certain time that shouldbe in a second state at the certain time according to the projectschedule (where the first state differs from the second state, where thedifference between the first state and the second state is at least aselect threshold, etc.), and so forth. In some examples, the analysis ofthe construction plan and/or the image data to identify discrepancybetween the construction plan and the construction site (for example, asdescribed above) may use information from the project schedule todetermine which discrepancies between the construction plan and theconstruction site are of importance at a selected time according to theproject schedule, to determine which discrepancies between theconstruction plan and the construction site are expected (and thereforeshould be, for example, ignored, treated differently, etc.) at aselected time according to the project schedule, to determine whichdiscrepancies between the construction plan and the construction siteare unexpected at a selected time according to the project schedule, andso forth.

In some examples, when the at least one electronic record comprises afinancial record associated with the construction site (such asfinancial records 625, financial records obtained by Step 920, etc.),Step 930 may identify at least one discrepancy between the financialrecord and the construction site. For example, the financial recordsand/or the image data may be analyzed to identify an object in theconstruction site that should not be in the construction site accordingto the financial record (for example, an object that was not paid for,was not ordered, that it's rental have not yet begun or have alreadyended, that is associated with an entity that should not be in theconstruction site according to the financial records, etc.), to identifyan object that should be in the construction site according to thefinancial records that is not in the construction site (for example, anobject that according to the financial records was paid for, wasordered, was delivered, was invoiced, was installed, is associated withan entity that should be in the construction site according to thefinancial records, etc.), to identify an object in the construction sitethat is in a first state at a certain time that should be in a secondstate at the certain time according to the financial records (forexample, where the first state differs from the second state, where thedifference between the first state and the second state is at least aselect threshold, etc., for example, where the work for changing thestate of the object to the second state was ordered, was billed, waspaid for, etc.), and so forth. In some examples, the analysis of theconstruction plan and/or the image data to identify discrepancy betweenthe construction plan and the construction site (for example, asdescribed above) may use information from the financial records todetermine which discrepancies between the construction plan and theconstruction site are of importance at a selected time according to thefinancial records (for example, have financial impact that is beyond aselected threshold), to determine which discrepancies between theconstruction plan and the construction site are not accurately reflectedin the financial records, and so forth. In some examples, the analysisof the progress record and/or the image data to identify discrepancybetween the progress record and the construction site (for example, asdescribed below) may use information from the financial records todetermine which discrepancies between the progress record and theconstruction site are of importance at a selected time according to thefinancial records (for example, have financial impact that is beyond aselected threshold), to determine which discrepancies between theprogress record and the construction site are not accurately reflectedin the financial records, and so forth.

In some examples, when the at least one electronic record comprises aprogress record associated with the construction site (such as progressrecords 630, progress records obtained by Step 920, etc.), Step 930 mayidentify at least one discrepancy between the progress record and theconstruction site. For example, the progress records and/or the imagedata may be analyzed to identify an object in the construction site thatshould not be in the construction site according to the progress record,to identify an object that should be in the construction site accordingto the progress records that is not in the construction site, toidentify an object in the construction site that is in a first statethat should be in a second state according to the progress records (forexample, where the first state differs from the second state, where thedifference between the first state and the second state is at least aselect threshold, etc.), to identify an action that is not reflected inthe image data but that is reported as completed in the progress record,to identify an action that is reflected in the image data but is notreported as complete in the progress record, and so forth. In someexamples, the analysis of the construction plan and/or the image data toidentify discrepancy between the construction plan and the constructionsite (for example, as described above) may use information from theprogress records to determine which discrepancies between theconstruction plan and the construction site are in contradiction to theinformation in the progress records, to determine which discrepanciesbetween the construction plan and the construction site are correctlyreflected at a selected time in the progress records, and so forth.

In some examples, when the at least one electronic record comprises anas-built model associated with the construction site (such as as-builtmodel 615, as-built model obtained by Step 920, etc.), Step 930 mayidentify at least one discrepancy between the as-built model and theconstruction site. For example, Step 930 may analyze the as-built modeland/or the image data to identify an object in the as-built model thatdoes not exist in the construction site, to identify an object in theconstruction site that does not exist in the as-built model, to identifyan object that has a specified location according to the as-built modeland is located at a different location in the construction site (forexample, to identify an object for which the discrepancy between thelocation according to the as-built model and the location in theconstruction site is above a selected threshold), to identify an objectthat should have a specified property according to the as-built modelbut has a different property in the construction site (some examples ofsuch property may include type of the object, location of the object,shape of the object, dimensions of object, color of the object,manufacturer of the object, type of elements in the object, setting ofthe object, technique of installation of the object, orientation of theobject, time of object installment, etc.), to identify an object thatshould be associated with a specified quantity according to the as-builtmodel but is associated with a different quantity in the constructionsite (some examples of such quantities may include size of the object,length of the object, number of elements in the object, etc.), and soforth.

In some embodiments, Step 940 may provide information (for example, to auser, to another process, to an external device, etc.) based, at leastin part, on the at least one discrepancy identified by Step 930. Forexample, in response to a first identified discrepancy, Step 940 mayprovide information (for example, to a user, to another process, to anexternal device, etc.), and in response to a second identifieddiscrepancy, the providence of the information by Step 940 may beforgone. In another example, in response to a first identifieddiscrepancy, Step 940 may provide first information, and in response toa second identified discrepancy, Step 940 may provide secondinformation, different from the first information, for example, to auser, to another process, to an external device, and so forth. In someexamples, Step 940 may provide information to a user as a visual output,audio output, tactile output, any combination of the above, and soforth. For example, Step 940 may provide the information to the user: bythe apparatus analyzing the information (for example, an apparatusperforming at least part of Step 930), through another apparatus (suchas a mobile device associated with the user, mobile phone 111, tablet112, and personal computer 113, etc.), and so forth. For example, theamount of information provided by Step 940, the events triggering theprovidence of information by Step 940, the content of the informationprovided by Step 940, and the nature of the information provided by Step940 may be configurable.

In some examples, Step 940 may present a presentation of at least partof the image data with an overlay presenting information based, at leastin part, on the at least one discrepancy identified by Step 930 (forexample, using a display screen, an augmented reality display system, aprinter, and so forth). For example, objects corresponding to theidentified discrepancies may be marked by an overlay. In anotherexample, information related to properties of the identifieddiscrepancies may be presented in conjunction with the depiction of theobjects corresponding to the identified discrepancies in the image data.For example, an overlay presenting desired dimensions of an object (suchas a room, a wall, a doorway, a window, a tile, an electrical box, etc.)may be presented over a depiction of the object, for example as textualinformation specifying the desired dimensions and/or the actualdimensions, as a line or a shape demonstrating the desired dimensions,and so forth. In another example, an overlay presenting desired locationof an object (such as a doorway, an electrical box, a pipe, etc.) may bepresented in conjunction with a depiction of the object, for example asan arrow pointing from the depiction of the object to the correctlocation, as a marker marking the correct location, as textualinformation detailing the offset in object location, and so forth. Inyet another example, an overlay presenting a desired object missing fromthe construction site may be presented over the image data, for examplein or next to the desired location for the object, with an indication ofthe type and/or properties of the desired object, and so forth. Inanother example, an overlay marking an object in the construction sitethat should not be in the construction site may be presented over ornext to the depiction of the object, for example including an X or asimilar mark over the object, including textual information explainingthe error, and so forth. In yet another example, an overlay marking anobject in the construction site that has properties different from somedesired properties may be presented over or next to the depiction of theobject, for example including a marking of the object, including textualinformation detailing the discrepancies in properties, and so forth.

In some examples, Step 940 may present a visual presentation of at leastpart of a construction plan with markings visually presentinginformation based, at least in part, on the at least one discrepancyidentified by Step 930 (for example, using a display screen, anaugmented reality display system, a printer, and so forth). For example,objects corresponding to the identified discrepancies may be marked inthe displayed construction plan. In another example, information relatedto properties of the identified discrepancies may be presented inconjunction with the depiction of the objects corresponding to theidentified discrepancies in the construction plan. In yet anotherexample, information may be presented as an overlay over thepresentation of the construction plan, for example in similar ways tothe overlay over the image data described above.

In some examples, Step 940 may present a visual presentation of at leastpart of a project schedule with markings visually presenting informationbased, at least in part, on the at least one discrepancy identified byStep 930 (for example, using a display screen, an augmented realitydisplay system, a printer, and so forth). For example, tasks in theproject schedules corresponding to the identified discrepancies may bemarked in the displayed project schedule. Moreover, information aboutthe identified discrepancies may be displayed in conjunction with themarked tasks. For example, the information about the identifieddiscrepancies may be displayed in conjunction to the marked task and mayinclude an amount of actual delay, an amount of predicted future delay,an amount of advance, construction errors associated with the task, andso forth.

In some examples, Step 940 may present a visual presentation of at leastpart of a financial record with markings visually presenting informationbased, at least in part, on the at least one discrepancy identified byStep 930 (for example, using a display screen, an augmented realitydisplay system, a printer, and so forth). For example, items in thefinancial records (such as payments, orders, bills, deliveries,invoices, purchase orders, etc.) corresponding to the identifieddiscrepancies may be marked in the displayed financial record. Moreover,information about the identified discrepancies may be displayed inconjunction with the marked items. For example, the information aboutthe identified discrepancies may be displayed in conjunction to themarked item and may include an amount of budget overrun, an amount ofpredicted future budget overrun, a financial saving, an inconsistency indates associated with the item, and so forth.

In some examples, Step 940 may present a visual presentation of at leastpart of a progress record with markings visually presenting informationbased, at least in part, on the at least one discrepancy identified byStep 930 (for example, using a display screen, an augmented realitydisplay system, a printer, and so forth). For example, items in theprogress record corresponding to the identified discrepancies may bemarked in the displayed progress record. Some examples of such items mayinclude an action that is not reflected in the image data but that isreported as completed in the progress record, an action that isreflected in the image data but is not reported as complete in theprogress record, and so forth. Moreover, information about theidentified discrepancies may be displayed in conjunction with the markeditems.

In some examples, Step 940 may present a visual presentation of at leastpart of an as-built model with markings visually presenting informationbased, at least in part, on the at least one discrepancy identified byStep 930 (for example, using a display screen, an augmented realitydisplay system, a printer, and so forth). For example, objectscorresponding to the identified discrepancies may be marked in thedisplayed as-built model. In another example, information related toproperties of the identified discrepancies may be presented inconjunction with the depiction of the objects corresponding to theidentified discrepancies in the as-built model. In yet another example,information may be presented as an overlay over the presentation of theas-built model, for example in similar ways to the overlay over theimage data described above.

In some examples, the information provided by Step 940 may comprisesafety data. For example, the at least one electronic record associatedwith a construction site obtained by Step 920 may comprise safetyrequirements associated with the construction site. Further, Step 930may analyze image data captured from a construction site (such as imagedata captured from the construction site using at least one image sensorand obtained by Step 710) to identify at least one discrepancy betweenthe safety requirements associated with the construction site and theconstruction site. Further, Step 940 may provide information based, atleast in part, on the at least one discrepancy between the safetyrequirements and the construction site identified by Step 930. Forexample, a type of scaffolds to be used (for example, at a specifiedlocation at the construction site) may be detailed in the safetyrequirements, while a different type of scaffolds (for example, lesssafe, incompatible, etc.) may be used in the construction site, asdepicted in the image data and identified by Step 930. Further, inresponse to the identification of the usage of the different type ofscaffolds by Step 930, Step 940 may provide information about the usageof a type of scaffolds incompatible with the safety requirements, mayvisually indicate the location of the incompatible scaffolds (forexample, in the image data, in a construction plan, in an as-builtmodel, etc.), and so forth.

In some examples, Step 930 may analyze image data (such as image datacaptured from the construction site using at least one image sensor andobtained by Step 710) and/or electronic records (such as the at leastone electronic record associated with a construction site obtained byStep 920) to compute a measure of the at least one discrepancyidentified by Step 930. For example, Step 930 may analyze the image dataand/or the electronic records using an artificial neural networkconfigured to compute measures of the discrepancies from image dataand/or electronic records. In another example, Step 930 may analyze theimage data and/or the electronic records using a machine learning modeltrained using training examples to compute measures of the discrepanciesfrom image data and/or electronic records. Further, the computed measureof a discrepancy may be compared with a selected threshold, and based ona result of the comparison, providing the information related to thediscrepancy by Step 940 may be withheld. For example, in response to afirst result of the comparison, Step 940 may provide the information,while in response to a second result of the comparison, providing theinformation may be delayed and/or forgone. For example, the at least onediscrepancy identified by Step 930 may comprise a discrepancy in aposition of an object between a construction plan and the constructionsite, the measure may include a length between the position according tothe construction plan and the position in the construction site, and thethreshold may be selected according to a legal and/or a contractualobligation associated with the construction site. In another example,the at least one discrepancy identified by Step 930 may comprise adiscrepancy in a quantity associated with an object (some examples ofsuch quantity may include size of the object, length of the object,dimensions of a room, number of elements in the object, etc.) between aconstruction plan and the construction site, the measure may include adifference between the quantity according to the construction plan andthe quantity in the construction site, and the threshold may be selectedaccording to a regulatory and/or a contractual obligation associatedwith the construction site. In yet another example, the at least onediscrepancy identified by Step 930 may comprise a discrepancy in a timethat an object is installed between a planned time of installationaccording to a project schedule and the actual time of installation inconstruction site according to the image data, the measure may include alength of the time difference, and the threshold may be selectedaccording to at least one float (the amount of time that a task in aproject schedule can be delayed without causing a delay) associated withthe task comprising the installation of the object in the projectschedule. In another example, the at least one discrepancy identified byStep 930 may comprise a discrepancy between a status of a task accordingto progress records and the status of the task in the construction site,and the measure may include a difference in the amount of units handledin the task (area covered in plaster, area covered with tiles, number ofelectrical boxes installed, etc.) between the amount according toprogress records and the amount in the construction site according tothe image data.

Consistent with the present disclosure, image data (such as image datacaptured from the construction site using at least one image sensor andobtained by Step 710) may be analyzed to detect at least one object inthe construction site, for example as described below in relation withStep 1120. Further, the image data may be analyzed to identify at leastone property of the at least one object (such as position, size, color,object type, etc.), for example as described below in relation with Step1120. In some examples, Step 940 may further provide information basedon the at least one property. For example, providing the information maybe further based on at least one position associated with the at leastone object (such as, an actual position of the object in theconstruction site, a position of a depiction of the object in the imagedata, a planned position for the object according to a constructionplan, etc.), for example by providing to the user an indicator of theposition, for example, as a set of coordinates, as an indicator on amap, as an indicator on a construction plan, as an indicator in anoverlay over a presentation of the image data, and so forth. In anotherexample, providing the information may be further based on a property ofthe object (such as size, color, object type, quality, manufacturer,volume, weight, etc.), for example by presenting the value of theproperty as measured from the image data, by presenting the plannedand/or required value (or range of values) for the property according tothe electronic records (for example, construction plan, financialrecords showing the manufacturer, as-built model, etc.), by presentingthe difference between the two, and so forth.

In some examples, the image data (such as image data captured from theconstruction site using at least one image sensor and obtained by Step710) may comprise one or more indoor images of the construction site,the at least one object detected by Step 1120 may comprise a pluralityof tiles paving an indoor floor, the at least one property determined byStep 1120 may comprise a number of tiles in the construction siteaccording to the image data, the discrepancy identified by Step 930 maycomprise a discrepancy between the number of tiles in the constructionsite according to the image data and the planned number of tilesaccording to the electronic records, and the information provided byStep 940 may comprise an indication about the discrepancy between thenumber of tiles in the construction site and the at least one electronicrecord. For example, the electronic record may comprise financialrecords comprising a number of tiles that were billed for, a number oftiles that were paid for, a number of tiles that were ordered, and soforth. In another example, the electronic record may comprise aconstruction plan comprising a planned number of tiles. In yet anotherexample, the electronic record may comprise a progress record comprisingthe number of tiles that were reported as installed in the constructionsite.

Consistent with the present disclosure, image data (such as image datacaptured from the construction site using at least one image sensor andobtained by Step 710) may be analyzed to identify at least oneconstruction error, for example using Step 1120 as described below.Further, Step 940 may provide an indication of the at least oneconstruction error, for example as described above. For example, animage depicting the construction error may be present to a user, forexample with a visual indicator of the construction error. In anotherexample, the location of the construction error may be indicated on amap, on a construction plan, on an as-build model, and so forth. In yetanother example, textual information describing the construction errormay be presented to the user. In some examples, the image data and/orthe electronic records may be further analyzed to identify a type of theat least one construction error. For example, the image data may beanalyzed using a machine learning model trained using training examplesto determine type of construction errors from images and/or electronicrecords. In another example, the image data may be analyzed using anartificial neural network configured to determine a type of constructionerrors from images and/or electronic records. Further, based, at leastin part, on the identified type of the at least one construction error,Step 940 may forgo and/or withhold providing at least part of theinformation. For example, in response to a first identified type of theat least one construction error, information may be provided to theuser, and in response to a second identified type of the at least oneconstruction error, Step 940 may forgo providing the information. Inanother example, in response to a first identified type of the at leastone construction error, Step 940 may provide first information to theuser, and in response to a second identified type of the at least oneconstruction error, Step 940 may provide second information differentfrom the first information to the user. In some examples, the image datamay be further analyzed to determine a severity associated with the atleast one construction error. For example, the image data and/or theelectronic records may be analyzed using a machine learning modeltrained using training examples to determine severity of constructionerrors from images and/or electronic records. In another example, theimage data may be analyzed using an artificial neural network configuredto determine a severity of construction errors from images and/orelectronic records. Further, based, at least in part, on the determinedseverity, Step 940 may forgo and/or withhold providing at least part ofthe information. For example, in response to a first determinedseverity, Step 940 may provide information to the user, and in responseto a second determined severity, Step 940 may forgo providing theinformation. In another example, in response to a first determinedseverity, Step 940 may provide first information to the user, and inresponse to a second determined severity, Step 940 may provide secondinformation different from the first information to the user.

Consistent with the present disclosure, position data associated with atleast part of the image data may be obtained, for example as describedabove with relation to Step 710. Further, Step 940 may provideinformation based, at least in part, on the obtained position data. Forexample, a portion of a construction plan and/or as-build modelcorresponding to the position data may be selected and presented to theuser (for example, the position data may specify a room and theconstruction plan and/or as-build model for the room may be presented,the position data may specify coordinates and a portion of theconstruction plan and/or as-build model comprising a locationcorresponding to the specified coordinates may be presented, and soforth). In another example, objects associated with the position data(for example, according to a construction plan) may be selected, andStep 940 may present information related to the selected objects (forexample, from objects database 605, construction plans 610, as-builtmodels 615, project schedules 620, financial records 625, progressrecords 630, safety records 635, and construction errors 640, etc.) tothe user.

Consistent with the present disclosure, time associated with at leastpart of the image data (such as capturing time, processing time, etc.)may be obtained. Further, Step 940 may provide information based, atleast in part, on the obtained time. For example, Step 940 may presentportions of a project schedule and/or progress records related to theobtained time. In another example, a project schedule and/or progressrecords may be analyzed to select objects related to the obtained time(for example, objects related to tasks that occur or should occur at orin proximity to the obtained time), and information related to theselected objects (for example, from objects database 605, constructionplans 610, as-built models 615, project schedules 620, financial records625, progress records 630, safety records 635, and construction errors640, etc.) may be presented to the user.

Consistent with the present disclosure, the image data obtained by Step710 may comprise at least a first image corresponding to a first pointin time and a second image corresponding to a second point in time, andthe elapsed time between the first point in time and the second point intime may be at least a selected duration (for example, at least an hour,at least one day, at least two days, at least one week, etc.). Further,Step 930 may analyze the image data for the identification of the atleast one discrepancy by comparing the first image with the secondimage. For example, differences between the images may be identifiedwith relation to a first object while no differences between the imagesmay be identified with relation to a second object, and Step 930 mayidentify a discrepancy when a progress record does not specify anymodification of the first object and/or when a progress record specifiesmodification of the second object. In another example, an identifieddifference may indicate that a new object was installed between thefirst point in time and the second point in time, and Step 930 mayidentify a discrepancy when a project schedule do not specify suchinstallation in the corresponding time interval.

Consistent with the present disclosure, data based on image datacaptured from at least one additional construction site may be obtained.Further, Step 940 may provide information based, at least in part, onthe obtained data, for example as described above. For example,information about the plurality of construction sites may be aggregated,as described below, statistics from the plurality of construction sitesmay be generated, and Step 940 may provide information based, at leastin part, on the generated statistics to the user. In another example,information from one construction site may be compared with informationfrom other construction sites, and Step 940 may provide informationbased, at least in part, on that comparison.

FIG. 10A is a schematic illustration of an example construction plan1000 consistent with an embodiment of the present disclosure. Forexample, construction plan 1000 may be stored in construction plans 610.Construction plan 1000 may include plans of objects, such as window1005, interior wall 1010, sink 1015, exterior wall 1020, and door 1025.As described above, Step 930 may identify discrepancies between theconstruction site and the construction plan.

In some examples, Step 930 may identify that window 1005 in theconstruction site is not according to construction plan 1000. Forexample, the position of window 1005 in the construction site may be notaccording to construction plan 1000. Further, the deviation in theposition of window 1005 may be calculated. In another example, the size(such as height, width, etc.) of window 1005 in the construction sitemay be not according to construction plan 1000. Further, the deviationin the size of window 1005 may be calculated. In yet another example,materials and/or parts of window 1005 in the construction site may benot according to construction plan 1000. In another example, window 1005may be missing altogether from the construction site, for example havinga wall instead. In yet another example, window 1005 may exist in theconstruction site but be missing altogether from construction plan 1000.In some examples, the calculated deviation may be compared with aselected deviation threshold. In some examples, information may beprovided to a user, for example using Step 940, based on thediscrepancies between window 1005 in the construction site andconstruction plan 1000, based on the calculated deviation, based on aresult of the comparison of the calculated deviation with the selecteddeviation threshold, and so forth.

In some examples, Step 930 may identify that interior wall 1010 in theconstruction site is not according to construction plan 1000. Forexample, the position of interior wall 1010 in the construction site maybe not according to construction plan 1000 (and as a result, an adjacentroom may be too small or too large). Further, the deviation in theposition of interior wall 1010 and/or in the size of the adjacent roomsmay be calculated. In another example, the size (such as height, width,thickness, etc.) of interior wall 1010 in the construction site may benot according to construction plan 1000. Further, the deviation in thesize of interior wall 1010 may be calculated. In yet another example,materials and/or parts of interior wall 1010 in the construction sitemay be not according to construction plan 1000. In another example,interior wall 1010 may be missing altogether from the construction site,for example having two adjacent rooms connected. In yet another example,interior wall 1010 may exist in the construction site but be missingaltogether from construction plan 1000, for example having a room splitinto two. In some examples, the calculated deviation may be comparedwith a selected deviation threshold. In some examples, information maybe provided to a user, for example using Step 940, based on thediscrepancies between interior wall 1010 in the construction site andconstruction plan 1000, based on the calculated deviation, based on aresult of the comparison of the calculated deviation with the selecteddeviation threshold, and so forth.

In some examples, Step 930 may identify that sink 1015 in theconstruction site is not according to construction plan 1000. Forexample, the position of sink 1015 in the construction site may be notaccording to construction plan 1000. Further, the deviation in theposition of sink 1015 may be calculated. In another example, the size ofsink 1015 in the construction site may be not according to constructionplan 1000. Further, the deviation in the size of sink 1015 may becalculated. In yet another example, materials and/or parts of sink 1015in the construction site may be not according to construction plan 1000.In another example, sink 1015 may be missing altogether from theconstruction site. In yet another example, sink 1015 may exist in theconstruction site but be missing altogether from construction plan 1000.In some examples, the calculated deviation may be compared with aselected deviation threshold. In some examples, information may beprovided to a user, for example using Step 940, based on thediscrepancies between sink 1015 in the construction site andconstruction plan 1000, based on the calculated deviation, based on aresult of the comparison of the calculated deviation with the selecteddeviation threshold, and so forth.

In some examples, Step 930 may identify that a pipe required for sink1015 is implemented incorrectly in the construction site. For example,an end of the pipe may be in an incorrect position in the constructionsite according to the position of sink 1015 in construction plan 1000Further, the deviation in the position of the end of the pipe may becalculated. In another example, the pipe in the construction site may beconnected to a wrong water source according to construction plan 1000.In yet another example, the pipe may be missing altogether from theconstruction site. In yet another example, the pipe may exist in theconstruction site but be missing altogether from construction plan 1000.In some examples, the calculated deviation may be compared with aselected deviation threshold. In some examples, information may beprovided to a user, for example using Step 940, based on thediscrepancies between the pipe in the construction site and constructionplan 1000, based on the calculated deviation, based on a result of thecomparison of the calculated deviation with the selected deviationthreshold, and so forth.

In some examples, Step 930 may identify that exterior wall 1020 in theconstruction site is not according to construction plan 1000. Forexample, the position of exterior wall 1020 in the construction site maybe not according to construction plan 1000 (and as a result, an adjacentroom may be too small or too large, connected wall may be too narrow ortoo wide, for example too narrow for door 1025, and so forth). Further,the deviation in the position of exterior wall 1020 and/or in the sizeof the adjacent room and/or in the size of connected walls may becalculated. In another example, the size (such as height, width,thickness, etc.) of exterior wall 1020 in the construction site may benot according to construction plan 1000. Further, the deviation in thesize of exterior wall 1020 may be calculated. In yet another example,materials and/or parts of exterior wall 1020 in the construction sitemay be not according to construction plan 1000. In another example,exterior wall 1020 may be missing altogether from the construction site,for example having a room connected to the yard. In yet another example,exterior wall 1020 may exist in the construction site but be missingaltogether from construction plan 1000, for example creating anadditional room. In some examples, the calculated deviation may becompared with a selected deviation threshold. In some examples,information may be provided to a user, for example using Step 940, basedon the discrepancies between exterior wall 1020 in the construction siteand construction plan 1000, based on the calculated deviation, based ona result of the comparison of the calculated deviation with the selecteddeviation threshold, and so forth.

In some examples, Step 930 may identify that door 1025 in theconstruction site is not according to construction plan 1000. Forexample, the position of door 1025 in the construction site may be notaccording to construction plan 1000. Further, the deviation in theposition of door 1025 may be calculated. In another example, the size(such as height, width, etc.) of door 1025 in the construction site maybe not according to construction plan 1000. Further, the deviation inthe size of door 1025 may be calculated. In yet another example,materials and/or parts of door 1025 in the construction site may be notaccording to construction plan 1000. In another example, door 1025 maybe missing altogether from the construction site, for example having awall instead. In yet another example, door 1025 may exist in theconstruction site but be missing altogether from construction plan 1000.In some examples, the calculated deviation may be compared with aselected deviation threshold. In some examples, information may beprovided to a user, for example using Step 940, based on thediscrepancies between door 1025 in the construction site andconstruction plan 1000, based on the calculated deviation, based on aresult of the comparison of the calculated deviation with the selecteddeviation threshold, and so forth.

FIG. 10B is a schematic illustration of an example image 1050 capturedby an apparatus consistent with an embodiment of the present disclosure.For example, image 1050 may depicts objects in a construction site, suchas electrical boxes 1055A, 1055B, 1055C, 1055D and 1055E, electricalwires 1060A, 1060B, and 1060C, and an unidentified box 1065. Asdescribed above, Step 930 may identify discrepancies between theconstruction site as depicted in image 1050 and construction planassociated with the construction site.

In some examples, Step 930 may identify that electrical boxes 1055A,1055B, 1055C, 1055D and 1055E in the construction site are not accordingto a construction plan associated with the construction site. Forexample, the position of electrical boxes 1055A, 1055B, 1055C, 1055D and1055E in the construction site may be not according to a constructionplan associated with the construction site. Further, the deviation inthe position of electrical boxes 1055A, 1055B, 1055C, 1055D and 1055Emay be calculated. In another example, the size (such as radius, depth,etc.) of electrical boxes 1055A, 1055B, 1055C, 1055D and 1055E in theconstruction site may be not according to a construction plan associatedwith the construction site. Further, the deviation in the size ofelectrical boxes 1055A, 10556, 1055C, 1055D and 1055E may be calculated.In yet another example, materials and/or parts and/or type of electricalboxes 1055A, 10556, 1055C, 1055D and 1055E in the construction site maybe not according to a construction plan associated with the constructionsite. In another example, at least one of additional electrical boxincluded in the construction plan may be missing altogether from theconstruction site. In yet another example, at least one of electricalboxes 1055A, 10556, 1055C, 1055D and 1055E may exist in the constructionsite but be missing altogether from a construction plan associated withthe construction site. In some examples, the calculated deviation may becompared with a selected deviation threshold. In some examples,information may be provided to a user, for example using Step 940, basedon the discrepancies between electrical boxes 1055A, 1055B, 1055C, 1055Dand 1055E in the construction site and a construction plan associatedwith the construction site, based on the calculated deviation, based ona result of the comparison of the calculated deviation with the selecteddeviation threshold, and so forth.

In some examples, Step 930 may identify that electrical wires 1060A,1060B, and 1060C in the construction site are not according to aconstruction plan associated with the construction site. For example,the position of electrical wires 1060A, 10606, and 1060C (or of an endpoint of electrical wires 1060A, 1060B, and 1060C) in the constructionsite may be not according to a construction plan associated with theconstruction site. Further, the deviation in the position of electricalwires 1060A, 10606, and 1060C may be calculated. In another example, thesize (such as length, diameter, etc.) of electrical wires 1060A, 1060B,and 1060C in the construction site may be not according to aconstruction plan associated with the construction site. Further, thedeviation in the size of electrical wires 1060A, 1060B, and 1060C may becalculated. In yet another example, materials and/or parts and/or typeof electrical wires 1060A, 1060B, and 1060C in the construction site maybe not according to a construction plan associated with the constructionsite. In another example, at least one of additional electrical wireincluded in the construction plan may be missing altogether from theconstruction site. In yet another example, at least one of electricalwires 1060A, 1060B, and 1060C may exist in the construction site but bemissing altogether from a construction plan associated with theconstruction site. In some examples, the calculated deviation may becompared with a selected deviation threshold. In some examples,information may be provided to a user, for example using Step 940, basedon the discrepancies between electrical boxes 1055A, 1055B, 1055C, 1055Dand 1055E in the construction site and a construction plan associatedwith the construction site, based on the calculated deviation, based ona result of the comparison of the calculated deviation with the selecteddeviation threshold, and so forth.

FIG. 11 illustrates an example of a method 1100 for updating recordsbased on construction site images. In this example, method 1100 maycomprise: obtaining image data captured from a construction site (Step710), analyzing the image data to detect objects (Step 1120), andupdating electronic records based on the detected objects (Step 1130).In some implementations, method 1100 may comprise one or more additionalsteps, while some of the steps listed above may be modified or excluded.For example, Step 1130 may be excluded from method 1100. In someimplementations, one or more steps illustrated in FIG. 11 may beexecuted in a different order and/or one or more groups of steps may beexecuted simultaneously and vice versa. For example, Step 1120 may beexecuted after and/or simultaneously with Step 710, Step 1130 may beexecuted after and/or simultaneously with Step 1120, and so forth.

Additionally or alternatively, Step 930 may identify a discrepancybetween electronic records and the construction site as depicted in theimage data, for example as described above, and in response Step 1130may update the electronic records according to the identifieddiscrepancy.

In some embodiments, Step 1120 may analyze image data (such as imagedata captured from the construction site using at least one image sensorand obtained by Step 710) to detect at least one object in theconstruction site and/or to determine properties of objects. Someexamples of such properties of objects may include type of object,position of object in the image data, position of the object in theconstruction site, size of the object, dimensions of the object, weightof the object, shape of the object, colors of the object, orientation ofthe object, state of the object, and so forth. In some examples, Step1120 may analyze the image data using a machine learning model trainedusing training examples to detect objects and/or to determine propertiesof objects from images. For example, some training examples may includean image depicting an object together with label detailing informationabout the depicted object such as the type of the object, position ofthe object in the image, properties of the object, and so forth. Othertraining examples may include images that do not depict objects fordetection, together with labels indicating that the images do not depictobjects for detection. In some examples, Step 1120 may analyze the imagedata using an artificial neural network configured to detect objectsand/or to determine properties of objects from images.

In some embodiments, Step 1130 may update at least one electronic recordassociated with the construction site based, at least in part, on the atleast one object detected by Step 1120 and/or properties of the at leastone object determined by Step 1120.

In some examples, Step 1120 may analyze the image data to identify atleast one position related to the at least one object detected by Step1120, and the update to the at least one electronic record may befurther based on the identified at least one position. In some examples,items and/or portions of the at least one electronic record associatedwith the identified at least one position may be selected, and theselected items and/or portions may be updated in the at least oneelectronic record, for example based on the at least one object detectedby Step 1120 and/or properties of the at least one object determined byStep 1120. For example, objects in database 605 may be selectedaccording to the identified at least one position, and the selectedobjects may be updated. In another example, portions of as-built model615 and/or construction plan 610 may be selected according to theidentified at least one position, and the selected portions may beupdated. In some examples, a record of a position associated with the atleast one object detected by Step 1120 may be updated in the at leastone electronic record according to the identified at least one position,for example a position of an object may be registered in an as-builtmodel 615, in database 605, and so forth. In some examples, theidentified at least one position related to the at least one object maybe compared with a position associated with the object in the at leastone electronic record (for example, with a position of the object inconstruction plan 610), and construction errors 640 may be updated basedon a result of the comparison (for example, registering a constructionerror in construction errors 640 when the difference in the position isabove a selected threshold, and forgoing registration of a constructionerror when the difference is below the selected threshold).

In some examples, Step 1120 may analyze the image data to identify atleast one property of the at least one object (such as position, size,color, object type, and so forth), and Step 1130 may update the at leastone electronic record based, at least in part, on the at least oneproperty. In some examples, records of the at least one electronicrecord associated with the identified at least one property may beselected, and Step 1130 may update the selected records in the at leastone electronic record, for example based on the at least one objectdetected by Step 1120 and/or properties of the at least one objectdetermined by Step 1120. For example, the selected record may beassociated with a specific object type (such as tile, electrical box,etc.), and the selected records may be updated (for example to accountfor the tiles or the electrical boxes detected in the image data). Insome examples, Step 1130 may update a record of a property associatedwith the at least one object detected by Step 1120 in the at least oneelectronic record according to the identified at least one property. Insome examples, the identified at least one property related to the atleast one object may be compared with a property associated with theobject in the at least one electronic record (for example, with aproperty of the object in construction plan 610), and Step 1130 mayupdate construction errors 640 based on a result of the comparison (forexample, registering a construction error in construction errors 640when the difference in the property is above a selected threshold, andforgoing registration of a construction error when the difference isbelow the selected threshold).

In some examples, the at least one electronic record associated with theconstruction site may comprise a searchable database, and Step 1130 mayupdate the at least one electronic record by indexing the at least oneobject in the searchable database. For example, the searchable databasemay be searched for a record related to the at least one object, inresponse to a determination that the searchable database includes arecord related to the at least one object, the record related to the atleast one object may be updated, and in response to a determination thatthe searchable database do not include a record related to the at leastone object, a record related to the at least one object may be added tothe searchable database. In some examples, such searchable database maybe indexed according to type of the objects, to properties of objects,to position of objects, to status of objects, to time the object wasidentified, to dimensions of the object, and so forth.

In some examples, when the image data comprises at least a first imagecorresponding to a first point in time and a second image correspondingto a second point in time (the elapsed time between the first point intime and the second point in time may be at least a selected duration,for example, at least an hour, at least one day, at least two days, atleast one week, etc.), Step 1130 may update the at least one electronicrecord based, at least in part, on a comparison of the first image andthe second image. For example, differences between the images may beidentified with relation to a first object while no differences betweenthe images may be identified with relation to a second object, and as aresult update to the at least one electronic record may be made withrelation to the first object, while updates related to the second objectmay be forwent. In another example, an identified difference mayindicate that a new object was installed between the first point in timeand the second point in time, and as result the installation of the newobject may be recorded in progress records 630 (for example with a timestamp associated with the first point in time and/or the second point intime), project schedule 620 may be updated to reflect the installationof the new object (for example, before the second point in time and/orafter the first point in time), as-build model 615 may be updated toreflect the installed new object, and so forth.

In some examples, the image data may comprise one or more indoor imagesof the construction site, the at least one object detected by Step 1120may comprise a plurality of tiles paving an indoor floor, the at leastone property determined by Step 1120 may comprise a number of tiles, andStep 1130 may update the at least one electronic record based, at leastin part, on the number of tiles. For example, Step 1130 may updatefinancial records 625 to reflect the number of tiles in the constructionsite, Step 1130 may update as-built model 615 with the number of tilesat selected locations in the construction site (room, balcony, selectedarea of a floor, selected unit, etc.), and so forth.

In some examples, the at least one electronic record may comprise atleast one as-built model associated with the construction site (such asas-built model 615), and Step 1130 may update to the at least oneelectronic record by modifying the at least one the as-built model. Forexample, an as-built model may be updated to include objects detected byStep 1120 (for example by analyzing images of the construction site), torecord a state and/or properties of objects in the as-built modelaccording to the state and/or properties of the objects in theconstruction site as determined by Step 1120 (for example by analyzingimages of the construction site), to position an object in the as-buildmodel according to the position of the object in the construction siteas determined by Step 1120 (for example by analyzing images of theconstruction site, according to the position of the image sensor thecaptured the images, etc.), and so forth.

In some examples, the at least one electronic record may comprise atleast one project schedule associated with the construction site (suchas project schedule 620), and Step 1130 may update the at least oneelectronic record by updating the at least one project schedule, forexample by updating at least one projected date in the at least oneproject schedule. For example, Step 1120 may analyze image data capturedat different points in time to determine a pace of progression, and Step1130 may update at least one projected finish date in the at least oneproject schedule based on the amount of remaining work in the task andthe determined pace of progression. For example, an analysis may showthat a first number of units were handled within a selected elapsedtime, and a pace of progression may be calculated by dividing the firstnumber of units by the selected elapsed time. Moreover, a remainingnumber of units to be handled in the task may be obtained, for examplefrom project schedule 620 and/or progress records 630. Further, theremaining number of units may be divided by the calculated pace ofprogression to estimate a remaining time for the task, and the projectedfinish date of the task may be updated accordingly. In another example,Step 1120 may analyze image data captured at a selected time todetermine that a task that should have started according to projectschedule 620 haven't yet started in the construction site. In response,Step 1130 may update projected dates associated with the task (such asprojected starting date, projected finish date, projected intermediatedates, and so forth). In yet another example, Step 1130 may updateprojected date in project schedule 620 (for example as described above),and may further update other dates in project schedule 620 that dependon the updated dates. For example, a first task may start only after asecond task is completed, and Step 1130 may update projected dates ofthe first task (such as the projected starting date, projected finishtime, etc.) after the projected finish date of the second task isupdated.

In some examples, the at least one electronic record may comprise atleast one financial record associated with the construction site (suchas financial record 625), and Step 1130 may update the at least oneelectronic record by updating the at least one financial record, forexample by updating at least one amount in the at least one financialrecord. For example, Step 1120 may analyze image data captured atdifferent points in time to determine a pace of progression, for exampleas described above, and Step 1130 may update at least one projectedfuture expense (for example, updating a projected date of the projectedfuture expense, updating a projected amount of the projected futureexpense, etc.) based on the determined pace of progression. In anotherexample, Step 1120 may analyze image data to determine that a task wasprogressed or completed, and in response to the determination, a paymentassociated with the task may be approved, placed for approval, executed,etc., and the financial records may be updated by Step 1130 accordingly.In yet another example, Step 1120 may analyze image data to determinethat a task was not progressed or completed as specified in anelectronic record (for example not progressed or completed as plannedaccording to project schedule 620, not progressed or completed asreported according to progress records 630, etc.), and in response tothe determination a payment associated with the task may be reduced,withheld, delayed, etc., and the financial records may be updated byStep 1130 accordingly. In another example, financial assessments may begenerated by analyzing image data depicting the construction site and/orelectronic records associated with the construction site (for example,using Step 1230 as described below), and Step 1130 may update financialrecords according to the generated financial assessments, for example byrecording the generated financial assessments in the financial records,by updating a financial assessment recorded in the financial recordsaccording to the generated financial assessments, in any other waydescribed below, and so forth.

In some examples, the at least one electronic record may comprise atleast one progress record associated with the construction site (such asprogress record 630), and Step 1130 may update the at least oneelectronic record by updating the at least one progress record, forexample by updating at least one progress status corresponding to atleast one task in the at least one progress record. For example, Step1120 may analyze image data to determine that a task was completed or acurrent percent of completion of the task, and Step 1130 may update atleast one progress status corresponding to the task in the at least oneprogress record according to the determination. In another example, Step1120 may analyze image data to determine that a task was not progressedor completed as specified in an electronic record (for example notprogressed or completed as planned according to project schedule 620,not progressed or completed as reported according to progress records630, etc.), and in response Step 1130 may record a delay in the at leastone progress record according to the determination.

In some examples, the at least one electronic record (for example, theat least one electronic record updated by Step 1130, the at least oneelectronic record obtained by Step 920, etc.) may comprise informationrelated to safety information. For example, image data (such as imagedata captured from the construction site using at least one image sensorand obtained by Step 710) may be analyzed to identify at least onesafety issue related to the at least one object detected by Step 1120,and Step 1130 may record information related to the at least one safetyissue in the at least one electronic record. For example, Step 1120 mayanalyze the image data to identify a type of scaffolds used in theconstruction site, the identified type of scaffolds may be compared withsafety requirements, and in response to a determination that the type ofscaffolds is incompatible with the safety requirements, and Step 1130may record a corresponding safety issue in safety records 635. Inanother example, Step 1120 may analyze the image data to detect a hangedobject loosely connected to the ceiling, and Step 1130 may record acorresponding safety issue in safety records 635.

In some examples, the at least one electronic record (for example, theat least one electronic record updated by Step 1130, the at least oneelectronic record obtained by Step 920, etc.) may comprise informationrelated to at least one construction error. For example, image data(such as image data captured from the construction site using at leastone image sensor and obtained by Step 710) may be analyzed to identifyat least one construction error related to the at least one objectdetected by Step 1120, and Step 1130 may record information related tothe at least one construction error in the at least one electronicrecord. For example, Step 1120 may analyze the image data to identify anobject installed incorrectly, and in response Step 1130 may record theincorrect installation of the object as a construction error inconstruction errors 640. In another example, Step 930 may identify adiscrepancy between electronic records (such as construction plan 610)and the construction site as depicted in the image data, for example asdescribed above, Step 1120 may identify a construction error based onthe identified discrepancy, for example as described above, and Step1130 may record the construction error identified by Step 930 inconstruction errors 640.

In some examples, Step 1130 may update the at least one electronicrecord associated with the construction site based, at least in part, ona time associated with the image data. For example, the image data maycomprise at least a first image corresponding to a first point in timeand a second image corresponding to a second point in time, Step 1130may update the at least one electronic record based, at least in part,on a comparison of the first image and the second image, as describedabove. In another example, Step 1120 may detect an object in the imagedata and/or determine properties of an object in an image data capturedat a particular time (such as a particular minute, a particular hour, aparticular date, etc.), and Step 1130 may record the detected objectand/or the determined properties of the object together with theparticular time in objects database 605. Other examples where the updateis based on a time associated with the image data are described above.

In some examples, Step 1130 may update the at least one electronicrecord associated with the construction site based, at least in part, ona position associated with the image data. For example, Step 1120 maydetect an object in the image data and/or determine properties of anobject in an image data captured at a particular location (such as aparticular unit, a particular room, from a particular position withinthe room, from a particular angle, at a particular set of coordinatesspecifying a location, etc.), and Step 1130 may record the detectedobject and/or the determined properties of the object together with theparticular location in objects database 605. Other examples where theupdate is based on a position associated with the image data and/or on aposition of objects depicted in the image data are described above.

Consistent with the present disclosure, image data (such as image datacaptured from the construction site using at least one image sensor andobtained by Step 710) may be analyzed to detect at least one object inthe construction site, for example as described above in relation withStep 1120. Further, the image data may be analyzed to identify at leastone property of the at least one object (such as position, size, color,object type, and so forth), for example as described above in relationwith Step 1120. The identified at least one property may be used toselect at least one electronic record of a plurality of alternativeelectronic records associated with the construction site. Step 1130 mayupdate the selected at least one electronic record, for example based onthe detected at least one object and/or the identified at least oneproperty. For example, the plurality of alternative electronic recordsmay be associated with different types of objects, and the type of theobject detected by Step 1120 may be used to select an electronic recordassociated with the type of the detected object of the plurality ofalternative electronic records. In another example, the plurality ofalternative electronic records may be associated with different regionsof the construction site (for example, different rooms, different units,different buildings, etc.), and the position of the object detected byStep 1120 may be used to select an electronic record associated with aregion corresponding to the position of the detected object of theplurality of alternative electronic records.

In some examples, the at least one electronic record (for example, theat least one electronic record updated by Step 1130, the at least oneelectronic record obtained by Step 920, etc.) may comprise informationbased on at least one image captured from at least one additionalconstruction site. For example, the at least one electronic record maycomprise information derived from image data captured from a pluralityof construction sites. Moreover, the information about the plurality ofconstruction sites may be aggregated, and statistics from the pluralityof construction sites may be generated. Further, information from oneconstruction site may be compared with information from otherconstruction sites. In some examples, such statistics and/or comparisonsmay be provided to the user. In some examples, pace of progression atdifferent construction sites may be measured from image data asdescribed above, the measured pace of progression at the differentconstruction sites may be aggregated in an electronic record (forexample, in a database), statistics about the pace of progression may begenerated and/or provided to a user, a pace of progression in oneconstruction site may be compared to pace of progression in otherconstruction sites, and so forth. In some examples, statistical modeltying properties of the construction sites to the pace of progressionmay be determined (for example, using regression models, usingstatistical tools, using machine learning tools, etc.) based on theaggregated measured pace of progression at the different constructionsites. Further, the statistical model may be used to predict a pace ofprogression for other construction sites from properties of the otherconstruction sites. Additionally or alternatively, the statistical modelmay be used to suggest modification to a construction site in order toincrease the pace of progression in that construction site. In someexamples, construction errors at different construction sites may beidentified from image data as described above, the identifiedconstruction errors at the different construction sites may beaggregated in an electronic record (for example, in a database),statistics about the construction errors may be generated and/orprovided to a user, construction errors in one construction site may becompared to construction errors in other construction sites, and soforth. In some examples, statistical model tying properties of theconstruction sites to construction errors may be determined (forexample, using regression models, using statistical tools, using machinelearning tools, etc.) based on the aggregated construction errors fromthe different construction sites. Further, the statistical model may beused to predict construction errors likely to occur at otherconstruction sites from properties of the other construction sites (forexample, together with a predict amount of construction errors).Additionally or alternatively, the statistical model may be used tosuggest modification to a construction site in order to avoid ordecrease construction errors in that construction site.

FIG. 12 illustrates an example of a method for generating financialassessments based on construction site images. In this example, method1200 may comprise: obtaining image data captured from a constructionsite (Step 710); obtaining electronic records associated with theconstruction site (Step 920); and generating financial assessments (Step1230). In some implementations, method 1200 may comprise one or moreadditional steps, while some of the steps listed above may be modifiedor excluded. For example, Step 920 may be excluded from method 1200. Insome implementations, one or more steps illustrated in FIG. 12 may beexecuted in a different order and/or one or more groups of steps may beexecuted simultaneously and vice versa. For example, Step 920 may beexecuted before and/or after and/or simultaneously with Step 710, Step1230 may be executed after and/or simultaneously with Step 710 and/orStep 920, and so forth.

In some embodiments, Step 1230 may analyze image data (such as imagedata captured from the construction site using at least one image sensorand obtained by Step 710) and/or at least one electronic record (such asat least one electronic record associated with the construction siteobtained by Step 920) to generate at least one financial assessmentrelated to the construction site. In one example, the financialassessment generated by Step 1230 may be recorded in financial records625. In another example, financial assessments in financial records 625may be updated according to the financial assessment generated by Step1230. In some examples, Step 1230 may analyze the image data and/or theat least one electronic record using a machine learning model trainedusing training examples to generate at least one financial assessmentfrom image data and/or electronic records. In some examples, Step 1230may analyze the image data and/or the at least one electronic recordusing an artificial neural network configured to generate at least onefinancial assessment from image data and/or electronic records.

In some examples, the image data may be analyzed to identify at leastone discrepancy between the at least one electronic record and theconstruction site, for example by Step 930 as described above, and Step1230 may use the identified at least one discrepancy to generate the atleast one financial assessment. For example, Step 930 may analyze theimage data to identify a delay with respect to a planned scheduleaccording to a project schedule as described above, and in response tothe identified delay Step 1230 may update a financial assessment ofprojected incomes associated with the construction site, Step 1230 mayupdate a financial assessment of required capital associated with theconstruction site, and so forth. In another example, Step 930 mayanalyze the image data to identify a divergence from a construction planas described above, and in response to the identified divergence Step1230 may update a valuation of the construction project, Step 1230 mayupdate an estimated risk associated with the construction site, and soforth. For example, a mathematical model of the projected incomesassociated with the construction site and/or of the required capitalassociated with the construction site and/or of the valuation of aconstruction project and/or of estimated risks associated with theconstruction site may use a formula or an algorithm that takes delaysand/or divergence from a construction plan as input, and Step 1230 mayuse the mathematical model to update the projected incomes associatedwith the construction site and/or the required capital associated withthe construction site and/or the valuation of a construction projectand/or estimated risks associated with the construction using theidentified delays and/or the identified divergence from the constructionplan.

In some examples, the image data may comprise at least a first imagecorresponding to a first point in time and a second image correspondingto a second point in time, the elapsed time between the first point intime and the second point in time may be at least a selected duration(for example, at least an hour, at least one day, at least two days, atleast one week, etc.), and Step 1230 may generate at least one financialassessment based, at least in part, on a comparison of the first imageand the second image. For example, the comparison may identify that aplurality of actions were performed in the construction site between thefirst point of time and the second point in time (some examples of suchactions may include installation of objects, advancement in a process,damaging an element of the construction site, etc.), and a financialassessment associated with the first point in time may be updatedaccording to the identified plurality of actions. In another example,the comparison may determine that fewer action than planned wereperformed in the construction site (for example, that no action wasperformed), a delay may be predicted as a response of the determination(or as described above), and the financial assessment may be updatedaccording to the predicted delay.

In some examples, the at least one electronic record may comprise aconstruction plan associated with the construction site, and Step 1230may use the construction plan to generate financial assessments. Forexample, an identified divergence from a construction plan may be usedto generate financial assessments as described above. In anotherexample, a mathematical model used for the financial assessment (such asa mathematical model of a risk related to a loan associated with theconstruction site, of a risk assessment related to an insurance policyassociated with the construction site, of a valuation associated withthe construction site, etc.) may use a function of properties of theconstruction plan (such as constructed area, bill of materials generatedusing the construction plan, etc.) as input factors.

In some examples, the at least one electronic record may comprise aproject schedule associated with the construction site, and Step 1230may use the project schedule to generate financial assessments. Forexample, an identified delay with respect to a planned scheduleaccording to a project schedule may be used to generate financialassessments as described above. In another example, a mathematical modelused for the financial assessment (such as a mathematical model of arisk related to a loan associated with the construction site, of a riskassessment related to an insurance policy associated with theconstruction site, of a valuation associated with the construction site,etc.) may use a function of properties of the project schedule (such asexpected date of completion, amount of concurrent tasks, etc.) as inputfactors.

In some examples, the at least one electronic record may comprise afinancial record associated with the construction site, and Step 1230may use the financial record to generate financial assessments. Forexample, unplanned expenses and/or delayed expenses in the financialrecord may be used to generate financial assessments. In anotherexample, a mathematical model used for the financial assessment (such asa mathematical model of a risk related to a loan associated with theconstruction site, of a risk assessment related to an insurance policyassociated with the construction site, of a valuation associated withthe construction site, etc.) may use a function of details from thefinancial records (such as total expenses to date, planned expenses,late payments, bill of materials, etc.) as input factors.

In some examples, the at least one electronic record may comprise aprogress record associated with the construction site, and Step 1230 mayuse the progress record to generate financial assessments. For example,at least one progress status from the progress records may be used togenerate financial assessments. In another example, a mathematical modelused for the financial assessment (such as a mathematical model of arisk related to a loan associated with the construction site, of a riskassessment related to an insurance policy associated with theconstruction site, of a valuation associated with the construction site,etc.) may use a function of details from the progress records (such asdelays, percent of completion of tasks, etc.) as input factors.

In some examples, Step 1230 may generate at least one financialassessment based, at least in part, on a position associated with atleast part of the image data. For example, Step 1120 may detect anobject in the image data and/or determine properties of an object in animage data captured at a particular location (such as a particular unit,a particular room, from a particular position within the room, from aparticular angle, at a particular set of coordinates specifying alocation, etc.) as described above, Step 1130 may update electronicrecords based on the detected object and/or the determined properties ofthe object together with the particular location as described above, andStep 1230 may use the updated electronic records to generate the atleast one financial assessment as described above. In another example, amathematical model used for the financial assessment (such as amathematical model of a risk related to a loan associated with theconstruction site, of a risk assessment related to an insurance policyassociated with the construction site, of a valuation associated withthe construction site, etc.) may use a function of information extractedfrom the image data (for example, as described above) together with theparticular location as input factors.

In some examples, Step 1230 may generate at least one financialassessment based, at least in part, on a time associated with at leastpart of the image data (for example, capturing time of the at least partof the image data was captured, a time of processing of the at leastpart of the image data, and so forth). For example, the image data maycomprise at least a first image corresponding to a first point in timeand a second image corresponding to a second point in time, and Step1230 may generate at least one financial assessment based, at least inpart, on a comparison of the first image and the second image asdescribed above. In another example, a mathematical model used for thefinancial assessment (such as a mathematical model of a risk related toa loan associated with the construction site, of a risk assessmentrelated to an insurance policy associated with the construction site, ofa valuation associated with the construction site, etc.) may use afunction of information extracted from the image data (for example, asdescribed above) together with the time associated with at least part ofthe image data as input factors.

In some examples, Step 1230 may generate at least one financialassessment comprising a risk assessment related to a loan associatedwith the construction site, for example as described above. In someexamples, Step 1230 may generate at least one financial assessmentcomprising a risk assessment related to an insurance policy associatedwith the construction site, for example as described above. In someexamples, Step 1230 may generate at least one financial assessmentcomprising a valuation associated with the construction site, forexample as described above. For example, the valuation may comprise avaluation after a completion of construction in the construction siteassociated with at least part of a constructed building built in theconstruction site.

In some examples, image data (such as image data captured from theconstruction site using at least one image sensor and obtained by Step710) and/or at least one electronic record (such as at least oneelectronic record associated with the construction site obtained by Step920) may be analyzed to update at least one parameter of a loanassociated with the construction site. For example, a risk assessmentrelated to a loan associated with the construction site may be generatedas described above, and the at least one parameter of the loan may beupdated based, at least in part, on the generated risk assessment. Inanother example, a valuation associated with the construction site maybe generated as described above, and the at least one parameter of theloan may be updated based, at least in part, on the generated valuation.

In some examples, image data (such as image data captured from theconstruction site using at least one image sensor and obtained by Step710) and/or at least one electronic record (such as at least oneelectronic record associated with the construction site obtained by Step920) may be analyzed to update at least one parameter of an insurancepolicy associated with the construction site. For example, a riskassessment related to an insurance policy associated with theconstruction site may be generated as described above, and at least oneparameter of the insurance policy may be updated based, at least inpart, on the generated risk assessment. In another example, a valuationassociated with the construction site may be generated as describedabove, and the at least one parameter of an insurance policy associatedwith the construction site may be updated based, at least in part, onthe generated valuation.

In some examples, Step 1120 may analyze the image data and/or the atleast one electronic record to detect at least one object in theconstruction site, for example as described above. Further, Step 1120may further analyze the image data and/or the at least one electronicrecord to identify at least one property of the at least one object, forexample as described above. Step 1230 may generate at least onefinancial assessment based, at least in part, on the identified at leastone property. For example, the image data may comprise one or moreindoor images of the construction site, the at least one object maycomprise a plurality of tiles paving an indoor floor, the at least oneproperty may comprise a number of tiles, and the generated at least onefinancial assessment may be based, at least in part, on the number oftiles. In another example, the image data may comprise one or moreindoor images of the construction site, the at least one object maycomprise a wall, the at least one property may comprise area and/orpercent of the wall covered by plaster, and the generated at least onefinancial assessment may be based, at least in part, on the area and/orpercent of the wall covered by plaster.

Consistent with the present disclosure, at least one previous financialassessment related to the construction site may be accessed. Further,the at least one previous financial assessment may be compared with theat least one financial assessment generated by Step 1230 to determine amagnitude of change. The magnitude of change may be compared with aselected threshold. In some examples, in response to a determinationthat the magnitude of change is above the selected threshold, anotification may be provided to a user, while in response adetermination that the magnitude of change is below the selectedthreshold, providing the notification to the user may be forgone. Insome examples, in response to a determination that the magnitude ofchange is above the selected threshold, a first notification may beprovided to a user, while in response a determination that the magnitudeof change is below the selected threshold, a second notificationdifferent from the first notification may be provided to the user.

FIG. 13 illustrates an example of a method 1300 for hybrid processing ofconstruction site images. In this example, method 1300 may comprise:obtaining image data captured from a construction site (Step 710), andanalyzing the image data to attempt to recognize object depicted in theimage data (Step 1320). In some examples, when the attempt to recognizethe object fails, method 1300 may present at least part of the imagedata to a user (Step 1330), and receive feedback related to the objectfrom the user (Step 1340). In some implementations, method 1300 maycomprise one or more additional steps, while some of the steps listedabove may be modified or excluded. For example, Step 1330 and/or Step1340 may be excluded from method 1300. In some implementations, one ormore steps illustrated in FIG. 13 may be executed in a different orderand/or one or more groups of steps may be executed simultaneously andvice versa. For example, Step 1320 may be executed after and/orsimultaneously with Step 710, Step 1330 may be executed after and/orsimultaneously with Step 1320, and so forth.

In some embodiments, Step 1320 may analyze image data (such as imagedata captured from the construction site using at least one image sensorand obtained by Step 710) to attempt to recognize at least one objectdepicted in the image data and/or to attempt to determine properties ofat least one object depicted in the image data. Some examples of suchproperties of objects may include type of object, position of object inthe image data, position of object in the construction site, size ofobject, weight of object, shape of object, colors of object, orientationof object, state of object, and so forth. In some examples, Step 1320may analyze the image data using a machine learning model trained usingtraining examples to attempt to recognize objects and/or to attempt todetermine properties of objects from images, for example as describedabove in relation to Step 1120. In one example, the machine learningmodel may provide an indication that the attempt to recognize objectsand/or that the attempt to determine properties of objects failed. Inanother example, the machine learning model may provide a confidencelevel associated with recognition of an object and/or with adetermination of properties of objects, the confidence level may becompared with a selected threshold, and the attempt may be considered asa failure when the confidence level is lower than a selected threshold.In some examples, Step 1120 may analyze the image data using anartificial neural network configured to attempt to recognize objectsand/or to attempt to determine properties of objects from images, and toprovide a failure indication in case of a failure to recognize objectsand/or a failure to determine properties of objects.

In some examples, in response to a failure of Step 1320 to successfullyrecognize the at least one object and/or to successfully determineproperties of the at least one object, Step 1330 may present at leastpart of the image data to a user (for example, using a display screen,an augmented reality display system, a printer, and so forth) and/orStep 1340 may receive a feedback related to the at least one object fromthe user (for example, through a user interface, using an input device,textually using a keyboard, through speech using a microphone and speechrecognition, as a selection of one or more alternative of a plurality ofalternatives presented to the user by Step 1330, and so forth). Forexample, the failure to successfully recognize the at least one objectmay comprise a recognition of the at least one object with a confidencelevel lower than a selected threshold. In some examples, the image datamay be analyzed to select the at least part of the image data that Step1330 presents to the user. For example, at least part of the image datathat depicts at least part of the object that Step 1320 failed torecognize and/or failed to determine its properties may be selected. Inanother example, a construction plan associated with the constructionsite may be used to select at least part of the image data correspondingto an object in the construction plan that Step 1320 failed tosuccessfully recognize or to successfully determine its properties.

In some examples, the failure of Step 1320 to successfully recognize theat least one object may comprise a successful recognition of a categoryof the at least one object and a failure to successfully recognize aspecific type within the category. Further, in response to the failureof Step 1320 to successfully recognize the at least one object, Step1330 may present information associated with the recognized category toa user alongside the at least part of the image data. For example, acategory may include “electrical box”, while specific type within thecategory may include “round electrical box”, “square electrical box”,“rectangular electrical box”, “shallow electrical box”, “weatherproofelectrical box”, “plastic electrical box”, “metal electrical box”, andso forth. In another example, a category may include “tile”, whilespecific type within the category may include “marble tile”, “ceramictile”, “terrazzo tile”, “granite tile”, “travertine tile”, “limestonetile”, and so forth. In yet another example, a category may include“pipe”, while specific type within the category may include “PEX pipe”,“PVC pipe”, “rigid copper pipe”, “ABS pipe”, “flexible copper tubing”,“galvanized steel pipe”, “cast iron pipe”, “water supply pipe”,“drainage pipe”, “electrical pipe”, and so forth.

In some examples, the failure of Step 1320 to successfully determineproperties of the at least one object may comprise a successfulrecognition of a type of the at least one object and a failure tosuccessfully determine at least one other property of the at least oneobject. Further, in response to the failure of Step 1320 to successfullydetermine at least one other property of the at least one object, Step1330 may present information associated with the recognized type to auser alongside the at least part of the image data. For example, thetype may include “electrical box”, and the at least one property mayinclude at least one of size, color, position, orientation, state,material, and so forth. In another example, the type may include “pipe”,and the at least one property may include at least one of end-point,size, length, color, position, state, material, and so forth. In yetanother example, the type may include “electrical wiring”, and the atleast one property may include at least one of end-point, length, color,position, state, and so forth.

In some examples, in response to the failure of Step 1320 tosuccessfully recognize the at least one object and/or to successfullydetermine properties of the at least one object, Step 1330 may presentto the user information associated with the construction site alongsidethe at least part of the image data. For example, at least a part of aconstruction plan (for example, at least a part of a construction plancorresponding to the presented at least part of the image data) may bepresented. In another example, at least a part of a progress record (forexample, at least a part of a progress record corresponding to the areaof the object) may be presented.

In some examples, in response to the failure of Step 1320 tosuccessfully recognize the at least one object and/or to successfullydetermine properties of the at least one object, Step 1330 may presentto the user information associated with the at least one object anddetermined by analyzing the image data alongside the at least part ofthe image data. For example, a size and/or a shape of the object may bedetermined from the image data and presented to the user. In someexamples, in response to the failure of Step 1320 to successfullyrecognize the at least one object and/or to successfully determineproperties of the at least one object, Step 1330 may present to the userinformation related to a position associated with the at least oneobject alongside the at least part of the image data. In some examples,in response to the failure of Step 1320 to successfully recognize the atleast one object and/or to successfully determine properties of the atleast one object, Step 1330 may present to the user information relatedto a position associated with at least a portion of the image dataalongside the at least part of the image data (for example, position ofthe camera when capturing the portion of the image data, position of atleast one item depicted in the portion of the image data, and so forth).In some examples, in response to the failure of Step 1320 tosuccessfully recognize the at least one object and/or to successfullydetermine properties of the at least one object, Step 1330 may presentto the user information related to a time associated with at least aportion of the image data alongside the at least part of the image data(for example, time the portion of the image data was captured, time theportion of the image data was recorded, and so forth).

In some examples, the attempt of Step 1320 to recognize the at least oneobject and/or to determine properties of the at least one object may bebased, at least in part, on a construction plan associated with theconstruction site. For example, a position of the at least one object inthe construction site (for example, as depicted in the image data) maybe used to select candidate objects from a construction plan (forexample, objects in proximity to a position in the construction plancorresponding to the position of the at least one object in theconstruction site), and the image data may be analyzed to try and selectan object of the candidate objects fitting the depiction of the objectin the image data (for example, selecting the most fitting object,selecting an object with a fitting score above a selected threshold, andso forth). In another example, a machine learning model trained usingtraining examples to attempt to recognize objects and/or to attempt todetermine properties of objects from images and construction plans maybe used as described above. In yet another example, an artificial neuralnetwork configured to attempt to recognize objects and/or to attempt todetermine properties of objects from images and construction plans maybe used as described above. Further, in response to the failure tosuccessfully recognize the at least one object, Step 1330 may presentinformation based on the construction plan to the user alongside the atleast part of the image data. For example, Step 1330 may present aportion of the construction plan corresponding to the location of the atleast one object in the image data to the user alongside the at leastpart of the image data. In another example, Step 1330 may present to theuser information from the construction plan related to objects matchinga suggested object type from the attempt to recognize the object.

In some examples, a suggested object type may be obtained from theattempt of Step 1320 to recognize the at least one object, for exampleas described above. One or more objects may be selected from theconstruction plan based on the location of the at least one object inthe image data, for example by selecting objects in proximity to aposition in the construction plan corresponding to the location of theat least one object in the image data. One or more types of the selectedone or more objects may be obtained, for example from the constructionplan. Further, the failure to successfully recognize the at least oneobject may be identified based, at least in part, on a mismatch betweenthe suggested object type and the one or more types of the selected oneor more objects.

In some examples, a suggested object type may be obtained from theattempt of Step 1320 to recognize the at least one object, for exampleas described above. One or more objects matching the suggested objecttype in the construction plan may be selected. One or more positionsspecified in the construction plan for the one or more objects matchingthe suggested object type in the construction plan may be obtained.Further, the failure to successfully recognize the at least one objectmay be identified based, at least in part, on a mismatch between atleast one position of the at least one object in the image data and theone or more positions specified in the construction plan.

In some examples, the attempt of Step 1320 to recognize the at least oneobject may be based, at least in part, on a project schedule associatedwith the construction site. For example, a machine learning modeltrained using training examples to attempt to recognize objects and/orto attempt to determine properties of objects from images and projectschedule may be used as described above. In another example, anartificial neural network configured to attempt to recognize objectsand/or to attempt to determine properties of objects from images andproject schedule may be used as described above. In yet another example,the failure to successfully recognize the at least one object maycomprise an identification of at least one discrepancy between arecognized at least one object according to the image data and theproject schedule. Further, in response to the failure to successfullyrecognize the at least one object, information based, at least in part,on the project schedule may be presented to the user alongside the atleast part of the image data. For example, Step 1330 may present aportion of the project schedule related to tasks corresponding to aposition of the at least one object. In another example, Step 1330 maypresent a portion of the project schedule related to tasks correspondingto a suggested object type from the attempt to recognize the object.

In some examples, the attempt of Step 1320 to recognize the at least oneobject may be based, at least in part, on a financial record associatedwith the construction site. For example, a machine learning modeltrained using training examples to attempt to recognize objects and/orto attempt to determine properties of objects from images and financialrecords may be used as described above. In another example, anartificial neural network configured to attempt to recognize objectsand/or to attempt to determine properties of objects from images andfinancial records may be used as described above. In yet anotherexample, the failure to successfully recognize the at least one objectmay comprise an identification of at least one discrepancy between arecognized at least one object and the financial record. Further, inresponse to the failure to successfully recognize the at least oneobject, information based, at least in part, on the financial record maybe presented to the user alongside the at least part of the image data.For example, Step 1330 may present a portion of the financial recordsrelated to the position of the at least one object. In another example,Step 1330 may present a portion of the financial records related totasks corresponding to a suggested object type from the attempt torecognize the object.

In some examples, the attempt of Step 1320 to recognize the at least oneobject may be based, at least in part, on a progress record associatedwith the construction site. For example, a machine learning modeltrained using training examples to attempt to recognize objects and/orto attempt to determine properties of objects from images and progressrecords may be used as described above. In another example, anartificial neural network configured to attempt to recognize objectsand/or to attempt to determine properties of objects from images andprogress records may be used as described above. In another example, thefailure to successfully recognize the at least one object may comprisean identification of at least one discrepancy between a recognized atleast one object and the progress record. Further, in response to thefailure to successfully recognize the at least one object, informationbased, at least in part, on the progress record may be presented to theuser alongside the at least part of the image data. For example, Step1330 may present a portion of the progress records related to theposition of the at least one object. In another example, Step 1330 maypresent a portion of the progress records related to tasks correspondingto a suggested object type from the attempt to recognize the object.

FIG. 14 is a schematic illustration of a user interface 1400 consistentwith an embodiment of the present disclosure. In some examples, Step1320 may analyze image 1050 captured by Step 710 in an attempt torecognize object 1065. Further, in response to a failure of Step 1320 torecognize object 1065, Step 1330 may present image 1405 to a user usinguser interface 1400. Image 1405 may comprise at least part of image 1050depicting object 1065. Further, user interface 1400 may comprise anoverlay over image 1405 emphasizing object 1065, such as emphasize box1410. Further, user interface 1400 may comprise a presentation of query1415 to the user requesting the user to identify object 1065. Step 1340may receive from the user an identified object type for object 1065through user interface 1400. In another example, user interface 1400 maycomprise a presentation of query to the user requesting the user toprovide a property of object 1065 (not shown), and Step 1340 may receivefrom the user a property of object 1065 through user interface 1400. Inyet another example, Step 1340 may receive from the user through userinterface 1400 an indication that the type of the object and/or theproperty of the object in unknown to the user.

FIG. 15 illustrates an example of a method 1500 for ranking usingconstruction site images. In this example, method 1500 may comprise:obtaining image data captured from a construction site (Step 710);analyzing the image data to detect elements associated with an entity(Step 1520); analyzing the image data to determine properties indicativeof quality and associated with the detected elements (Step 1530); andrank the entity (Step 1540). In some implementations, method 1500 maycomprise one or more additional steps, while some of the steps listedabove may be modified or excluded. For example, Step 1540 may beexcluded from method 1500. In some implementations, one or more stepsillustrated in FIG. 15 may be executed in a different order and/or oneor more groups of steps may be executed simultaneously and vice versa.For example, Step 1520 may be executed after and/or simultaneously withStep 710, Step 1530 may be executed after and/or simultaneously withStep 1520, Step 1540 may be executed after and/or simultaneously withStep 1530, and so forth.

In some embodiments, Step 1520 may analyze image data (such as imagedata captured from the construction site using at least one image sensorand obtained by Step 710) to detect at least one element depicted in theimage data and associated with an entity. In some examples, the at leastone element may include an element built and/or manufactured and/orinstalled and/or supplied by the entity. For example, Step 1520 mayanalyze objects database 605 and/or project schedule 620 and/orfinancial records 625 and/or progress records 630 to identify elementsbuilt and/or manufactured and/or installed and/or supplied by theentity, and analyze the image data to detect the identified elements,for example as described above. In some examples, the at least oneelement detected by Step 1520 may include an element built and/ormanufactured and/or installed and/or supplied by a second entity andaffected by a task performed by the entity. For example, image data frombefore and after the performance of the task may be analyzed to identifyelements that their state and/or condition changed, for example asdescribed above. In some examples, the at least one element detected byStep 1520 may be selected of a plurality of alternative elementsdetected in the image data, for example based on the entity. Forexample, an analysis of the image data may detect a number of elements(for example, a number of electrical boxes, a number of walls, etc.), ananalysis of the electronic records may indicate that the entity isrelated to a strict subset of the detected elements (for example,analysis of objects database 605 and/or project schedule 620 and/orfinancial records 625 and/or progress records 630 may indicate that onlya strict subset of the detected elements were built and/or manufacturedand/or installed and/or supplied by the entity), and the strict subsetof elements may be selected of the detected elements.

In some embodiments, Step 1530 may analyze the image data to determineat least one property indicative of quality and associated with the atleast one element. For example, a machine learning model may be trainedusing training example to determine properties indicative of quality andassociated with elements from image data, and Step 1530 may analyze theimage data using the trained machine learning model to determine the atleast one property indicative of quality and associated with the atleast one element. In another example, an artificial neural network maybe configured to determine properties indicative of quality andassociated with elements from image data, and Step 1530 may analyze theimage data using the artificial neural network to determine the at leastone property indicative of quality and associated with the at least oneelement. In some examples, the image data may comprise at least a firstimage corresponding to a first point in time and a second imagecorresponding to a second point in time, the elapsed time between thefirst point in time and the second point in time may be at least aselected duration (for example, at least an hour, at least one day, atleast two days, at least one week, etc.), and Step 1530 may determinethe at least one property indicative of quality based, at least in part,on a comparison of the first image and the second image. For example,the first image and the second image may be compared to determine aproperty of the curing process of concrete as described above. Inanother example, the first image and the second image may be compared todetermine a property of a pace of progression of a task, as describedabove. In yet another example, the first image and second image may becompared to determine a change in a state of an object, as describedabove, and the property may be determined based on the change of thestate, for example determining a first value of the property when thestate change from a first state to a second state and determining asecond value of the property when the state change from a first state toa third state.

In some embodiments, Step 1540 may use the at least one propertyindicative of quality determined by Step 1530 to generate a ranking ofthe entity. In some example, Step 1540 may generate a ranking comprisingone or more scores. Examples of such scores may include discrete scoresuch as “excellent”, “good”, “average” and “poor”; a numerical score;and so forth. Some examples of such scores may include a score for workpace, a score for completion of tasks on time, a score for delays, ascore for quality of work, a score for not harming unrelated elements inthe construction site, a score for compatibility with other elements inthe construction site, and so forth. For example, the at least oneproperty may indicate a work pace when performing tasks related to theentity (for example, “fast”, “average” and “slow”; a number of unitshandled within a selected time; etc.), and the calculated score mayinclude a weighted average of the work pace for the different tasks, amode of the work pace for the different tasks, and so forth. In anotherexample, the at least one property may indicate that a first portion ofthe tasks related to the entity were completed on time, a second portionof the tasks related to the entity were minorly delayed, and a thirdportion of the tasks related to the entity were delayed significantly,and a score for completion of tasks on time and/or a score for delaysmay be computed as a function of the ratio of the first, second andthird portions of the tasks of all the tasks related to the entity, as afunction of the actual delay times, as a function of the actual delaytime as a ratio of the planned time for each task, as a function of theactual delay time as a ratio of the entire length of performing eachtask, and so forth. Some examples of such function may include aweighted average of the delays or the ratio of the delays, a cumulativescore that adds positive values for tasks completed on time and negativevalues for delayed tasks (for example, for delays beyond a selectedthreshold), and so forth. In yet another example, the at least oneproperty may indicate a quality of work related to one or more objectsand/or tasks related to the entity, and the calculated score may includea weighted average of the quality of work for the different objectsand/or tasks, a cumulative score that adds positive values for objectsand/or tasks with good quality of work and negative values for objectsand/or tasks with poor quality of work, and so forth. In anotherexample, the at least one property may indicate that an object and/ortask related to the entity harmed another element at the constructionsite and/or was incompatible with another element and/or task in theconstruction site, and a score associated with the entity for notharming unrelated elements in the construction site and/or forcompatibility with other elements in the construction site may bereduced due the indication that an object and/or task related to theentity harmed another element at the construction site and/or wasincompatible with another element and/or task in the construction site.In some example, Step 1540 may generate a ranking of a first entity asbetter in at least one respect than a second entity. For example, Step1540 may generate a first score for the first entity and a second scorefor the second entity as described above, and when the first score ishigher than the second score rank the first entity as better than thesecond entity. In another example, a machine learning model may betrained using training examples to select a more compatible entity to atask of alternative entities using at least one property indicative ofquality, and Step 1540 may use the trained machine learning model togenerate a ranking of a first entity as better in at least one respectthan a second entity, for example by selecting the more compatibleentity according to the machine learning model as the better one.

In some examples, the image data may comprise one or more indoor imagesof the construction site, the at least one element of Step 1520 and/orStep 1530 may comprise at least one wall built by the entity, and the atleast one property may comprise a quantity of plaster applied to the atleast one wall. In some cases, the plaster may be applied by a differententity and still be indicative of the quality of the wall built by theentity, for example as more plaster may indicate a need to smoothdepressions and/or indentations in the wall. In some examples, Step 1530may analyze the image data to determine the quantity of plaster appliedto the at least one wall. For example, the amount of plaster applied tothe at least one wall may be estimated by comparing a depth image of thewall before applying the plaster to a depth image of the wall afterapplying the plaster, and a volume of the plaster may be estimatedaccording to the changes between the depth images. In another example,the amount of plaster applied to the at least one wall may be estimatedby a machine learning model trained using training examples to estimateamount of plaster from a 2D image of a wall before applying the plasterand a 2D image of the wall after applying the plaster. In some examples,Step 1540 may use the determined quantity of plaster applied to the atleast one wall to generate the ranking of the entity. For example, theranking of the entity may be lower when the amount of plaster applied tothe at least one wall is greater, for example by reducing the rankingaccording to the amount of plaster, by calculating the ranking using ascore function that is monotonically decreasing in the amount ofplaster, and so forth.

In some examples, the at least one element Step 1520 and/or Step 1530may comprise a room built by the entity. Further, Step 1530 may analyzethe image data to determine one or more dimensions of the room, forexample using a machine learning model trained using training examplesto determine dimensions of a room from image data, using an artificialneural network configured to determine dimensions of a room from imagedata, by measuring the dimensions in 3D images of the room, and soforth. Further, Step 1540 may use the determined one or more dimensionsof the room to generate the ranking of the entity. For example, the oneor more dimensions may be compared with desired dimensions of the room(for example, according to a construction plan), and the ranking of theentity may be lower when the discrepancy between the determineddimensions of the room and the desired dimensions of the room is larger,for example by reducing the ranking according to the amount ofdiscrepancy, by calculating the ranking using a score function that ismonotonically decreasing in the discrepancy, and so forth.

In some examples, Step 1530 may analyze the image data to identify signsof water leaks associated with the at least one element (such as a waterleak from a pipe, a water leak from an outside wall, a water leak from aceiling, etc.), for example using a machine learning model trained usingtraining examples to identify signs of water leaks from image data,using an artificial neural network configured to identify signs of waterleaks from image data, and so forth. Further, Step 1540 may use theidentified signs of water leaks to generate the ranking of the entity.For example, the ranking of the entity may be decreased when signs ofwater leaks are identified.

In some examples, Step 1530 may determine the at least one propertybased, at least in part, on at least one discrepancy between aconstruction plan associated with the construction site and theconstruction site, for example, based on at least one discrepancyidentified by Step 930 between the construction plan and theconstruction site as described above. For example, Step 930 may identifyan object in the construction plan that does not exist in theconstruction site as described above, and in response Step 1530 maydetermine the level of completeness of a task and/or the compliance toguidelines (for example, guidelines specified in the construction plan)when performing the task. In another example, Step 930 may identify anobject that has a specified location according to the construction planand is located at a different location in the construction site asdescribed above, and in response Step 1530 may determine the complianceto the construction plan related to the installation of the object. Inyet another example, Step 930 may identify an object that should have aspecified property according to the construction plan but has adifferent property in the construction site as described above, such asa different manufacturer, and in response Step 1530 may determine thatthe quality of materials used is below the specified quality specifiedin the construction plan.

In some examples, Step 1530 may determine the at least one propertybased, at least in part, on at least one discrepancy between a projectschedule associated with the construction site and the constructionsite, for example, based on at least one discrepancy identified by Step930 between the project schedule and the construction site as describedabove. For example, Step 930 may identify a discrepancy between adesired state of the construction site at a selected time according tothe project schedule and the state of the actual construction site atthe selected time as depicted in the image data as described above, andin response Step 1530 may determine an insufficient pace of work.

In some examples, Step 1530 may determine the at least one propertybased, at least in part, on at least one discrepancy between a financialrecord associated with the construction site and the construction site,for example, based on at least one discrepancy identified by Step 930between the financial record and the construction site as describedabove. For example, Step 930 may identify an object in the constructionsite that has a first property while the object should have a differentproperty according to the financial records (for example, differentmodel, different manufacturer, different size, etc.), and in responseStep 1530 may determine the supply to be inadequate.

In some examples, Step 1530 may determine the at least one propertybased, at least in part, on at least one discrepancy between a progressrecord associated with the construction site and the construction site,for example, based on at least one discrepancy identified by Step 930between the progress record and the construction site as describedabove. For example, Step 930 may identify an action that is notreflected in the image data but that is reported as completed in theprogress record, and in response Step 1530 may determine thatsupervision level is inadequate. In another example, Step 930 mayidentify an action that is reflected in the image data but that is notreported in the progress record, and in response Step 1530 may determinethat the reporting level is inadequate.

In some examples, Step 1540 may generate the ranking using informationbased, at least in part, on at least one image captured from at leastone additional construction site. For example, information from oneconstruction site may be compared with information from otherconstruction sites, and the ranking may include a ranking relative toother construction sites (for example, “above average”, “average”,“below average”, “1.6 standard deviations above mean”, and so forth). Inanother example, an entity may be associated with a plurality ofconstruction sites (such as a manufacturer producing products used at aplurality of construction sites, a supplier supplying products to aplurality of construction sites, a subcontractor building and/orinstalling elements at a plurality of construction sites, and so forth),and the ranking of the entity may be based on elements associated withthe entity from the plurality of construction sites.

Consistent with the present disclosure, the at least one elementdetected by Step 1520 may be further associated with a first technique(such as installation technique, building technique, drying technique,and so forth), and the ranking generated by Step 1540 may be associatedwith the entity and the first technique. For example, the techniqueassociated with an element may be specified in a database. In anotherexample, the image data may be analyzed to determine the techniqueassociated with the element, for example using a machine learning modeltrained using training examples to determine the technique associatedwith an element. In yet another example, Step 1520 may select elementsassociated with a selected technique of a plurality of alternativeelements. Further, Step 1520 may analyze the image data to detect anadditional group of at least one element depicted in the image data andassociated with the entity and a second technique, for example asdescribed above. Step 1530 may further analyze the image data todetermine an additional group of at least one property indicative ofquality and associated with the additional group of at least oneelement. Further, Step 1540 may use the additional group of at least oneproperty to generate a second ranking of the entity related to thesecond technique, for example as described above.

Consistent with the present disclosure, the at least one elementdetected by Step 1520 may be associated with a first group of one ormore additional elements, and the ranking generated by Step 1540 may beassociated with the entity and the first group. For example, in image1050, electrical box 1055D may be associated with electrical wire 1060Cand vice versa, for example due to connected functionality. In anotherexample, in image 1700, doorway 1755 may be associated with electricalbox 1760 and vice versa, for example due to proximity between the two.Further, Step 1520 may analyze the image data to detect an additionalgroup of at least one element depicted in the image data and associatedwith the entity and a second group of one or more additional elements,for example as described above. Step 1530 may further analyze the imagedata to determine an additional group of at least one propertyindicative of quality and associated with the additional group of atleast one element, for example as described above. Further, Step 1540may use the additional group of at least one property to generate asecond ranking of the entity related to the second group of one or moreadditional elements, for example as described above. In yet anotherexample, an element (such as a pipe, a wire, a box, a tile, etc.) may bepositioned adjunct and/or within to a surface (such as a wall, a floor,etc.), and therefore may be associated with the surface. Further, afirst ranking may be based on elements associated with a wall andtherefore the first ranking may be associated with walls, while a secondranking may be based on elements associated with a floor and thereforethe second ranking may be associated with floors.

Consistent with the present disclosure, the at least one elementdetected by Step 1520 may be further associated with a second entity,and the ranking generated by Step 1540 may be associated with the entityand the second entity. For example, the first entity may include amanufacturer of an element and the second entity may include asubcontractor installing the element. In another example, the firstentity may include a person building a wall and the second entity mayinclude a person plastering the wall. Further, Step 1520 may analyze theimage data to detect an additional group of at least one elementdepicted in the image data and associated with the entity and a thirdentity, for example as described above. Step 1530 may further analyzethe image data to determine an additional group of at least one propertyindicative of quality and associated with the additional group of atleast one element, for example as described above. Further, Step 1540may use the additional group of at least one property to generate asecond ranking of the entity related to the third entity, for example asdescribed above.

FIG. 16 illustrates an example of a method 1600 for annotation ofconstruction site images. In this example, method 1600 may comprise:obtaining image data captured from a construction site (Step 710);obtaining construction plan associated with the construction site (Step1620); analyzing the construction plan to identify a region of the imagedata corresponding to an object (Step 1630); presenting the image datawith an indication of the identified region (Step 1640); presenting aquery related to the object (Step 1650); receiving a response to thequery (Step 1660); and using the response to update electronic recordassociated with the construction site (Step 1670). In someimplementations, method 1600 may comprise one or more additional steps,while some of the steps listed above may be modified or excluded. Forexample, Step 1650 and/or Step 1660 and/or Step 1670 may be excludedfrom method 1600. In some implementations, one or more steps illustratedin FIG. 16 may be executed in a different order and/or one or moregroups of steps may be executed simultaneously and vice versa. Forexample, Step 1620 and/or Step 1630 may be executed before and/or afterand/or simultaneously with Step 710, and so forth.

Consistent with the present disclosure, image data associated with aconstruction site, such as image data captured from the constructionsite using at least one image sensor, may be obtained, for example byusing Step 710 as described above. Further, Step 1620 may obtain atleast one construction plan associated with the construction site (suchas construction plan 610) and including information related to anobject, for example by using Step 920 as described above. In someembodiments, Step 1630 may analyze the at least one construction planobtained by Step 1620 to identify a first region of the image datacorresponding to the object. For example, the at least one constructionplan may include a specified position for the object, such as a unit, aroom, a surface (such as a wall, a ceiling, a floor, etc.), a regionwithin the surface, position within the surface, a set of coordinates,and so forth. Further, Step 1630 may identify a first region of theimage data corresponding to the specified position for the object in theconstruction plan. For example, portions of the image data may beassociated with different positions, such as units, rooms, surfaces(such as a wall, a ceiling, a floor, etc.), regions within the surfaces,positions within the surfaces, range of coordinates, coordinates, and soforth, and Step 1630 may identify a first region of the image dataincluding the specified position for the object in the construction planand/or in proximity to that specified position. In another example, theimage data captured by Step 710 may be correlated with the constructionplan using an image registration algorithm, and Step 1630 may identify afirst region of the image data correlated to an area including theobject in the construction plan. In some examples, the informationrelated to the object in the construction plan obtained by Step 1620 mayinclude a planned location for the object, and Step 1630 may identify afirst region of the image data that may include a region of the imagedata corresponding to the planned location for the object, for exampleas described above.

In some examples, Step 1630 may analyze the image data (for example, inaddition to the at least one construction plan) to identify the firstregion of the image data corresponding to the object. In some examples,the construction plan may specific a general position of the object.Further, an analysis of the image data may identify one or morecandidate regions within the general position, and one of the one ormore candidate regions may be selected as the first region of the imagedata corresponding to the object. For example, the construction plan mayspecific the general position of the object as a particular wall, ananalysis of the depiction of the particular wall in the image data mayidentify one of the one or more candidate regions corresponding toirregularities in the pixel data depicting the walls (for example,different colors, different texture, etc.), and at least one of thecandidate regions may be selected as the first region of the image data,for example based on a height, based on size, based on shape, etc. Inanother example, the construction plan may specific the general positionof the object as a particular room, an analysis of the depiction of theparticular room may detect a floor and a wall, for example as describedabove, and based on the type of object (for example, “floor drainage”)the candidate region may be selected to be the region depicting thefloor in the image data. In some examples, an image analysis of theimage data (for example using Step 1320 as described above) may identifya region of the image data that depicts the object with some probability(for example, a probability higher than a first selected thresholdand/or lower than a second selected threshold), and the identifiedregion may be selected as the first region (for example, in response tothe probability being higher than the first selected threshold and/orlower than the second selected threshold).

In some examples, Step 1630 may use information based on an analysis ofsecond image data captured from the construction site before thecapturing of the image data from the construction site (for example, atleast an hour before, at least one day before, at least a week before,at least a month before), for example in addition to the at least oneconstruction plan, to identify the first region of the image datacorresponding to the object. In some examples, an image analysis of thesecond image data (for example using Step 1120 as described above) mayidentify a region of the second image data that depicts the object, anda region of the image data corresponding to that region of the secondimage data (for example, based on image registration results) may beselected as the first region.

Step 1640 may present at least part of the image data to a user with anindication of the first region of the image data identified by Step 1630as corresponding to the object, for example using a display screen, anaugmented reality display system, a printer, and so forth. In someexamples, the indication of the first region may include an overlay overthe presented image data. Such overlay may include an arrow pointing tothe first region, a bounding shape (such as a bounding circle, boundingrectangular box, bounding polygon, bounding free line, etc.), markingsof boundaries around the first region, marking of the center of thefirst region, marking of an interior point or area within the firstregion, and so forth. In some examples, the indication of the firstregion may include a mask of the first region. The mask may be presentednext to the image data, over the image data, and so forth. In someexamples, the indication of the first region may include a presentationof the first region of the image data using first display parameters(such as color scheme, intensity, etc.) while displaying other parts ofthe image data with different display parameters.

In some embodiments, Step 1650 may present a query related to the objectto the user, for example together with the presentation of Step 1640,for example visually, audibly, textually, using a display screen, usingan augmented reality display system, a printer, audio speakers, and soforth. In some examples, the query may be related to the object and/orthe image data and/or the identified first region. For example, Step1650 may present a query about the type of the object, possibly togetherwith a text box allowing the user to type in the type of object and/orwith a presentation of plurality of alternative object types that theuser may select from. In another example, Step 1650 may present a queryabout a property of the object (such as state, position, orientation,shape, color, dimensions, manufacturer, type of installation, etc.),possibly together with a text box allowing the user to type in the valueof the property and/or with a presentation of plurality of alternativevalues for the property that the user may select from. In some examples,several indications of several regions and/or several queries may bepresented together.

In some embodiments, Step 1660 may receive a response to the query ofStep 1660 from the user and/or inputs from the user. For example, thereceived response and/or inputs may be related to the object and/or theimage data and/or the identified first region. For example, the receivedresponse and/or inputs may be received through a user interface, usingan input device, textually using a keyboard, through speech using amicrophone and speech recognition, as a selection of one or morealternatives (for example, of a plurality of alternatives presented tothe user by Step 1650), and so forth. Some examples of such receivedresponse and/or inputs are described below.

In some embodiments, Step 1670 may use the response and/or the inputsreceived from the user by Step 1660 to update information associatedwith the object in at least one electronic record associated with theconstruction site. For example, the response and/or the inputs receivedfrom the user may indicate that the object is not in the regionidentified by Step 1630, and in response Step 1670 may remove the objectfrom objects database 605 and/or record an indication that the object isnot in the region identified by Step 1630 in region identified by Step1630, may update as-built model 615 by removing the object from an areaof as-built model 615 corresponding to the region identified by Step1630, may update project schedule 620 to reflect a delay deduced fromthe absent of the object as described above, may update financialrecords 625 based on the absent of the object as described above, updateprogress record 630 to reflect that a task associated with the object isnot completed, may update construction error 640 to reflect aconstruction error related to an absent of the object and/or to anincorrect location of the object, and so forth. In another example, theresponse and/or the inputs received from the user may indicate that theobject is in the region identified by Step 1630, and in response Step1670 may add a record of the object to objects database 605 (forexample, with an indication of the position of the object as a positionin the region identified by Step 1630), may update as-built model 615 byadding the object to an area of as-built model 615 corresponding to theregion identified by Step 1630, may update project schedule 620 and/orupdate progress record 630 to reflect a task completion deduced from thepresent of the object as described above, may update financial records625 based on the present of the object as described above, may updateconstruction error 640 to reflect a construction error related to thepresent of the object, and so forth. In yet another example, theresponse and/or the inputs received from the user may indicate that theobject is at a particular state and/or has a specified property, and inresponse Step 1670 may record the particular state and/or the specifiedproperty of the object in objects database 605, may update as-builtmodel 615 by modifying a representation of the object in the as-builtmodel 615 according to the particular state and/or the specifiedproperty, may update project schedule 620 and/or update progress record630 to reflect a task progression deduced from the particular stateand/or the specified property, may update financial records 625 based onthe particular state and/or the specified property, may updateconstruction error 640 to reflect a construction error related to theparticular state and/or the specified property, and so forth.

In some examples, the at least one construction plan associated with theconstruction site and obtained by Step 1620 may include informationrelated to a plurality of alternative objects, and at least oneelectronic project schedule associated with the construction site may beanalyzed to select the object of Step 1630 of the plurality ofalternative objects. In some examples, the project schedule may indicateexpected installation dates for the plurality of alternative objects,and object corresponding to a selected time range may be selected. Forexample, the selected time range may be selected based on a firstcapturing time of the image data and/or second capturing time of imagedata of a previously processed past image data, for example by selectinga time range approximately starting with the second capturing timeand/or approximately ending at the first capturing date, by selecting atime range including a selected time duration before the first capturingtime, by selecting a time range including a selected time duration afterthe first capturing time, and so forth. In another example, the selectedtime range may be selected based on a current time, for example byselecting a time range including a selected time duration before thecurrent time, by selecting a time range including a selected timeduration after the current time, and so forth. Further, any combinationof the above time ranges may be selected. In some examples, the projectschedule may include an indication of active tasks at the capturing timeof the image data and/or the current time, and objects related to theactive tasks may be selected of the plurality of alternative objects.

Consistent with the present disclosure, Step 1650 may present a query ofwhether the object is depicted in the identified first region of theimage data, for example as described above. Step 1660 may receive anindication of whether the object is depicted in the identified firstregion of the image data from the user, for example in response to thequery, for example as described above. Further, Step 1670 may use thereceived indication of whether the object is depicted in the identifiedfirst region of the image data to update at least one electronic recordassociated with the construction site. For example, Step 1670 may usethe received indication of whether the object is depicted in theidentified first region of the image data to update at least oneelectronic as-built model associated with the construction site, forexample as described above.

Consistent with the present disclosure, Step 1660 may receive anindication of at least one location corresponding to the object withinthe identified first region of the image data from the user. Further,Step 1670 may use the received indication of at least one locationcorresponding to the object to update at least one electronic recordassociated with the construction site. For example, the receivedindication of at least one location corresponding to the object may beused to update at least one electronic as-built model associated withthe construction site, for example by adding the object to a location ofthe as-built model corresponding to the indicated at least one location,by setting a location of an object that already exists in the as-builtmodel to the indicated at least one location, and so forth.

Consistent with the present disclosure, Step 1650 may present a query ofa construction stage associated with the object to a user. Step 1660 mayreceive an indication of the construction stage associated with theobject from a user, for example in response to the query. Step 1670 mayuse the received indication of the construction stage associated withthe object to update at least one electronic record associated with theconstruction site. For example, Step 1670 may use the receivedindication of the construction stage associated with the object toupdate at least one electronic progress record associated with theconstruction site, for example by updating a status of a task associatedwith the object according to the received indication of the constructionstage. In another example, Step 1670 may use the received indication ofthe construction stage associated with the object to update at least onetime indication associated with a future task in at least one electronicproject schedule associated with the construction site, for example whenthe received indication of the construction stage represent a delay in atask with respect to a plan according to the project schedule, and thedelay to that task may suggest delays to future tasks due to inner-tasksrelationships.

Consistent with the present disclosure, Step 1650 may present a query ofa quantity associated with the object to the user. Step 1660 may receivean indication of quantity associated with the object from the user, forexample in response to the query. Step 1670 may use the receivedindication of quantity associated with the object to update at least oneelectronic record associated with the construction site. For example,Step 1670 may use the received indication of quantity associated withthe object to update at least one electronic financial record associatedwith the construction site. For example, the object may include tiles,the quantity may include number of tiles, and the number of tiles may beused to update the financial record as described above. In anotherexample, the object may include a wall, the quantity may include area ofthe wall covered with plaster and/or amount of plaster used, and thearea of the wall covered with plaster and/or amount of plaster used maybe used to update the financial record, for example by updatinginformation based on a bill of materials and/or by updating a completionpercent of a task.

Consistent with the present disclosure, Step 1650 may present a query ofa state associated with the object to the user. Step 1660 may receive anindication of the state associated with the object from the user, forexample in response to the query. In some examples, Step 1670 may usethe received indication of the state associated with the object toupdate at least one electronic record associated with the constructionsite. For example, the received indication of the state associated withthe object may be used to identify at least one construction errorassociated with the object, for example as described above, and theidentified at least one construction error associated with the objectmay be used to update the at least one electronic record associated withthe construction site, such as records of construction errors 640 in adatabase. In some examples, the received indication of the stateassociated with the object may be used to identify at least one safetyissue associated with the object (for example, a “loosely connected”state may indicate a safety issue, as described above, and so forth).Further, the identified at least one safety issue associated with theobject may be used to update the at least one electronic recordassociated with the construction site, such as records of safety records635 in a database.

Consistent with the present disclosure, the at least one constructionplan associated with the construction site and obtained by Step 1620 mayfurther include information related to a second object. Step 1630 mayfurther analyze the at least one construction plan to identify a secondregion of the image data corresponding to the second object. Step 1640may present at least part of the image data to a user with an indicationof the identified second region of the image data corresponding to thesecond object. For example, the presentation of the indication of theidentified region of the image data corresponding to the object and thepresentation of the indication of the identified second region of theimage data corresponding to the second object may be at least partiallyconcurrent (for example, the indications of the two regions may bepresented on the same image, two different images each with one of thetwo indications of the regions may be present next to each other, and soforth). In another example, the presentation of the indication of theidentified region of the image data corresponding to the object and thepresentation of the indication of the identified second region of theimage data corresponding to the second object may be nonconcurrent.

Consistent with the present disclosure, in response to an indicationreceived from the user by Step 1660 that the object is depicted in theidentified first region of the image data, Step 1670 may make a firstupdate to the at least one electronic record associated with theconstruction site, for example as described above. Consistent with thepresent disclosure, Step 1630 may analyze the at least one constructionplan to identify a second region of the image data corresponding to theobject. For example, the identified second region may include at leastpart of the identified first region. In another example, the identifiedsecond region may include the identified first region entirely. In yetanother example, the identified second region may include no part of theidentified first region. For example, in response to an indicationreceived from the user that the object is not depicted in the identifiedfirst region of the image data, Step 1630 may select a second region ofthe image data corresponding to the object, for example by extending theregion of the image data originally selected by Step 1630, by selectinganother region from a plurality of alternative regions originallyconsidered by Step 1630, and so forth. In response to an indicationreceived from the user that the object is not depicted in the identifiedfirst region of the image data, Step 1640 may present at least part ofthe image data to a user with an indication of the identified secondregion of the image data corresponding to the object, for example asdescribed above. Further, Step 1650 may present a second query ofwhether the object is depicted in the identified second region of theimage data to the user, for example as described above. Step 1660 mayreceive an indication that the object is depicted in the identifiedsecond region of the image data from the user, for example in responseto the second query. In response to the indication that the object isnot depicted in the identified first region of the image data and to theindication that the object is depicted in the identified second regionof the image data, Step 1670 may make a second update to the at leastone electronic record associated with the construction site, for exampleas described above, and the second update may differ from the firstupdate. For example, any update made by Step 1670 that is made accordingto the first region in response to an indication received from the userby Step 1660 that the object is depicted in the identified first regionof the image data (as described above), may be made according to thesecond region in response to an indication that the object is notdepicted in the identified first region of the image data and to theindication that the object is depicted in the identified second regionof the image data.

Consistent with the present disclosure, in response to an indicationreceived from the user by Step 1660 that the object is not depicted inthe identified first region of the image data, method 1600 may causecapturing of additional image data from the construction site. Forexample, method 1600 may create a task in project schedule 620 for thecapturing of the additional image data from the construction site. Inanother example, method 1600 may transmit a signal configured to causeat least one image sensor to capture the additional image data from theconstruction site. In yet another example, the additional image data mayinclude the region identified by Step 1630. In another example, theadditional image data may include an alternative location of the object.In yet another example, the additional image data may be captured atleast selected time duration after the capturing of the image datapresented by Step 1650. In another example, the additional image datamay be obtained and/or captured using Step 710. In yet another example,the method 1600 may be repeated with the additional image data.

FIG. 17 is a schematic illustration of an example image 1700 captured byan apparatus consistent with an embodiment of the present disclosure.For example, image 1700 may depicts objects in a construction site, suchas doorway 1755, electrical box 1760, a pair of electrical boxes 1765,table 1770, and so forth. As described above, Step 1630 may analyze aconstruction plan and/or image 1700 to identify one or more regions ofthe image 1700 corresponding to any of the above objects. For example,Step 1630 may identify region 1705 as corresponding to doorway 1755, mayidentify region 1710 as corresponding to electrical box 1760, mayidentify regions 1715 and 1720 as corresponding to the pair ofelectrical boxes 1765, may identify region 1725 as corresponding to anobject occluded by table 1770 (the occluded object is not shown), and soforth. For any of the above objects and corresponding identifiedregions, Step 1640 may present image 1700 and/or a part of image 1700including the corresponding identified region, together with anindication of the identified region as described above. Further, Step1650 may present a query related to the object and/or to thecorresponding identified region, as described above. Some examples ofsuch queries may include a query of whether the object is within theregion, such as “is there a doorway in region 1705”, “is there anelectrical box in region 1710”, “is there an electrical box in region1715”, “is there an electrical box in region 1720”, “is there anelectrical box in region 1725”, and so forth. Some possible responsesthat Step 1660 may receive in return to such queries may include anindication of whether the object is with the region (for example,entirely, partially, or not at all, such as “the object is entirelywithin the region”, “the object is partly within the region”, “theobject is not in the region”, etc.), an indication that the object isnot within the region but near the region (for example, “the object isnear the region”), an indication that such determination cannot be madepossibly together with an indication of the reason that suchdetermination cannot be made (for example, “impossible to determine ifthe object is within the region”, “impossible to determine if the objectis within the region due to poor image quality”, “impossible todetermine if the object is within the region due to occlusions”, etc.),and so forth. Some other examples of queries that Step 1650 may presentmay include queries about the location of the object within the region,such as “what is the location of an object with a region”, “what is thelocation of the doorway in region 1705”, “what is the location of theelectrical box in region 1710”, “what is the location of the electricalboxes in region 1720”, and so forth. In response to such queries, theuser may provide an indication of the location of the object (forexample, marking a pixel within the object, marking an area within theobject, for example using scribbles, marking the boundaries of theobject, for example by using a bounding box, by using a bounding shape,by marking corners of the boundaries, etc., drawing a mask of theobject, and so forth), may indicate that the object is not in theregion, and so forth. Some other examples of queries that Step 1650 maypresent may include queries about a quantity related to the objects inthe region, such as dimensions, surface area, number of items, volume,weight, “how many electrical boxes are in region 1705”, “how manyelectrical boxes are in region 1720”, and so forth. In response to suchqueries, the user may provide an indication of quantity (such as numberof items, “no electrical box”, “one electrical box”, “two electricalboxes”, etc., dimensions, estimation of distance, “about two meters”,estimation of surface area, “about one square”, estimated volume,“between 10 and 15 cc”, estimated weight, “about 140 grams”, and soforth. Some other examples of queries that Step 1650 may present mayinclude a query about a properties (such as dimensions, shape, color,state, type, etc.) of an object in the region, such as “is there a doorin doorway 1755”, “what is the construction stage of electrical box1760”, “is the wall in region 1720 plastered”, and so forth. In someexamples, after receiving an indication that electrical boxes 1765 arenot in region 1715, Step 1640 may present region 1720 to the user andStep 1650 may present a query of whether electrical boxes 1765 are inregion 1720.

What is claimed is:
 1. A method for hybrid processing of constructionsite images, the method comprising: obtaining image data captured from aconstruction site using at least one image sensor; analyzing the imagedata to attempt to recognize at least one object depicted in the imagedata; obtaining a suggested object type from the attempt to recognizethe at least one object; based on a location of the at least one objectin the image data, selecting one or more objects in a construction planassociated with the construction site; obtaining one or more types ofthe selected one or more objects; identifying a failure to successfullyrecognize the at least one object based on a mismatch between thesuggested object type and the one or more types of the selected one ormore objects; and in response to the identification of the failure tosuccessfully recognize the at least one object: presenting at least partof the image data to a user; and receiving a feedback related to the atleast one object from the user.
 2. The method of claim 1, furthercomprising presenting to the user information based on the constructionplan alongside the at least part of the image data.
 3. The method ofclaim 1, further comprising: preprocessing the image data using atransformation function to obtain a transformed image comprising atleast a convolution of the image data; and analyzing the transformedimage to attempt to recognize the at least one object depicted in theimage data.
 4. The method of claim 1, further comprising: obtaining asuggested object type from the attempt to recognize the at least oneobject; selecting one or more objects matching the suggested object typein the construction plan; obtaining one or more positions specified inthe construction plan for the one or more objects matching the suggestedobject type in the construction plan; and further basing theidentification of the failure to successfully recognize the at least oneobject based on a mismatch between at least one position of the at leastone object in the image data and the one or more positions specified inthe construction plan.
 5. The method of claim 1, wherein the attempt torecognize the at least one object is based on a project scheduleassociated with the construction site, and further comprising presentingto the user information based on the project schedule alongside the atleast part of the image data.
 6. The method of claim 1, wherein theidentification of the failure to successfully recognize the at least oneobject is further based on an identification of at least one discrepancybetween a recognized at least one object and a project scheduleassociated with the construction site.
 7. The method of claim 1, whereinthe attempt to recognize the at least one object is based on a financialrecord associated with the construction site, and further comprisingpresenting to the user information based on the financial recordalongside the at least part of the image data.
 8. The method of claim 1,wherein the identification of the failure to successfully recognize theat least one object is further based on an identification of at leastone discrepancy between a recognized at least one object and a financialrecord associated with the construction site.
 9. The method of claim 1,wherein the attempt to recognize the at least one object is based on aprogress record associated with the construction site, and furthercomprising presenting to the user information based on the progressrecord alongside the at least part of the image data.
 10. The method ofclaim 1, wherein the identification of the failure to successfullyrecognize the at least one object is further based on an identificationof at least one discrepancy between a recognized at least one object anda progress record associated with the construction site.
 11. The methodof claim 1, wherein the identification of the failure to successfullyrecognize the at least one object is further based on a recognition ofthe at least one object with a confidence level lower than a selectedthreshold.
 12. The method of claim 1, wherein the identification of thefailure to successfully recognize the at least one object is furtherbased on a successful recognition of a category of the at least oneobject and a failure to successfully recognize a specific type withinthe category, and further comprising presenting to the user informationassociated with the recognized category alongside the at least part ofthe image data.
 13. The method of claim 1, further comprising analyzingthe image data to select the at least part of the image data.
 14. Themethod of claim 1, further comprising presenting to the user informationassociated with the construction site alongside the at least part of theimage data.
 15. The method of claim 1, further comprising presenting tothe user, alongside the at least part of the image data, informationassociated with the at least one object and determined by analyzing theimage data.
 16. The method of claim 1, further comprising presenting tothe user information related to a position associated with the at leastone object and determined by analyzing the image data alongside the atleast part of the image data.
 17. The method of claim 1, furthercomprising presenting to the user information related to a positionassociated with at least a portion of the image data alongside the atleast part of the image data.
 18. The method of claim 1, furthercomprising presenting to the user information related to a timeassociated with at least a portion of the image data alongside the atleast part of the image data.
 19. A system for hybrid processing ofconstruction site images, the system comprising: at least one imagesensor configured to capture image data from a construction site; and atleast one processor configured to: analyze the image data to attempt torecognize at least one object depicted in the image data; obtain asuggested object type from the attempt to recognize the at least oneobject; based on a location of the at least one object in the imagedata, select one or more objects in a construction plan associated withthe construction site; obtain one or more types of the selected one ormore objects; identify a failure to successfully recognize the at leastone object based on a mismatch between the suggested object type and theone or more types of the selected one or more objects; and in responseto the identification of the failure to successfully recognize the atleast one object: present at least part of the image data to a user; andreceive a feedback related to the at least one object from the user. 20.A non-transitory computer readable medium storing data and computerimplementable instructions for carrying out a method for hybridprocessing of construction site images, the method comprising: obtainingimage data captured from a construction site using at least one imagesensor; analyzing the image data to attempt to recognize at least oneobject depicted in the image data; obtaining a suggested object typefrom the attempt to recognize the at least one object; based on alocation of the at least one object in the image data, selecting one ormore objects in a construction plan associated with the constructionsite; obtaining one or more types of the selected one or more objects;identifying a failure to successfully recognize the at least one objectbased on a mismatch between the suggested object type and the one ormore types of the selected one or more objects; and in response to theidentification of the failure to successfully recognize the at least oneobject: presenting at least part of the image data to a user; andreceiving a feedback related to the at least one object from the user.